Achieving success

When considering performance, there is typically more focus on failure than there is on success. Failures are often noticeable, well-remembered one-time events that generate high emotions and can affect many people.  There is the idea that teams learn from our mistakes and failures, and that in order to improve teams must fail publicly and painfully. It is believed that significant successes must somehow be rooted in overcoming past misfortunes and failures.

Success can be loud and come with a lot of fanfare as well, but the fanfare of success tends to fade faster.  New challenges pop up and emotions go away quickly because the team must get back to work, as success tends to bring in more work and generate additional challenges.  

As a result, success appears to play a smaller role in overall evaluation than failure.  That is both unfortunate and counter-productive.  

It is unfortunate, because failures are not necessarily what makes us learn and improve going forward. While occasionally there is learning that is an outcome of failure, the same learning can also happen without experiencing the fail.  A lot of failures are unavoidable, but nevertheless result in blame (or self-blame) and reduce motivation.

Focus on failures is also counter-productive, because failure is hardly ever the enemy. Failures come from experiments, from aspiring to reach farther, from ambition.  Statistically, it takes a number of failures to achieve significant success.  Without these failures, the success is either impossible, or a lot less likely.

Success is not the opposite of failure, rather, it is a significantly different happening.  Success often comes with more work and more responsibility, creates additional challenges, and pushes the team to work harder.  Success requires active learning and deliberate practice, and there is a need to learn still more to handle challenges brought on by the achievement.  Success encourages ambition and experimenting, and boosts motivation.

Success is the main driver of future success. It is time to focus more on success, and less on failure, when considering team performance. 

Deciding the architecture

Agile project starts by gathering an Agile team, deciding on a project and specific deliverables to be built first, and setting up the needed infrastructure. 

But before the team goes about creating the infrastructure, several questions about the system architecture need to be addressed.  What infrastructure will be needed, and what skills the team must have, greatly depends on the proposed system architecture and technology stack.

Who decides system architecture is the “elephant in the room” issue of Agile projects.  Many smaller design decisions will happily “emerge” as the team goes through the daily work of building the project iteration after iteration. Still, there is a number of large-scale, long-lasting choices about system architecture, that must be decided at the start of the project, when relatively little is known and the team has the least amount of experience on the project.

Who should be making these decisions? The typical answer appears to be The Architect, someone high up in the ivory tower, someone who makes no mistakes by producing little to none tangible, and thus imperfect, results. Sometimes that person is very good, and in other cases they just get by.  As it turns out, making the very best initial architectural decisions for a given project is not particularly important for the project's success, and making passable architectural decisions is fairly easy and fool-proof. 

List of common architectural patterns is very small, and almost all projects fit nicely into that tiny set.  Even if an inappropriate pattern is picked, the team will be fighting, and winning, over the inappropriate pattern to create a working implementation.  If the system becomes successful despite poorly designed architecture, and fighting the system setup becomes a serious obstacle to evolving the product, it will get re-written with a less inappropriate architecture shortly.

P.S. 

Sydney Opera House is one of the most beautiful and distinctive buildings in the world. Its architecture went through a dozen iterations of design, and the finished structure exhibits terrible acoustics - a big problem for an opera house and performance venue.  The building took 10 years longer than planned, and went 1,357% over budget. Of course, the project was, and still is, a resounding success.

Scrum as a series of feedback cycles

• Daily scrum: every 24 hours, within the Scrum team. Feedback on technical details.

• Sprint review: every 1 to 3 weeks, within the product team. Feedback on requirements, features delivered, and implementation details.

• Retro: every 1 to 3 weeks, or, better yet, on as-needed basis, within the Scrum team. Feedback on communication, processes, resources.

• Backlog grooming: every 1-2 days or even more often, within the product team. Feedback on features delivered, technical details communicated, and user feedback.

• Sprint planning: not a feedback cycle on anything… why is it a part of Scrum?

A key to the kingdom

Agile talks about emergent architecture and building vertical slices of functionality. It is typically understood that all efforts must go toward creating valuable user-facing pieces, as opposed to delivering software layers that cannot be used until all other layers are in place.  Unfortunately, this approach is often taken as “all we need is the visible parts”, i.e. the parts that the user gets to see and click on, or that push the data to be displayed.

When I first encountered a large DB supporting a well-used, living application with no indexes or foreign keys, I thought it was a rare oversight.  Then I encountered a few more database setups just like it, with large and heavily used tables and relationships, but neither indexes nor keys defined.  In all cases, users were complaining about slow response times, and administrators remembered more than one instance of the application becoming non-responsive under heavy load as DB requests timed out en masse.

Other developers I talked to also mentioned that they worked with reasonably large DBs that were missing indexes and foreign keys. A number of people, working in different companies on unrelated projects, mentioned that their engineering managers or software architects absolutely refused all suggestions to create even primary keys on tables “since nobody sees them anyway”. Several developers talked about applications being re-implemented, in part, because of poor response times caused by frequent timeouts of DB requests.

Vertical slices of user-visible functionality must include “the plumbing” that allows the application to handle the expected load. “Emergent architecture” is not complete without at least basic considerations of technical quality. A project cannot succeed without understanding what it takes to deliver user value in a production situation – and that typically includes multiple users working simultaneously.  Building in proper DB infrastructure is a simple example of such considerations, the work that is not visible but nevertheless valuable to the user.

When the math is off

How Developers Rate Their Own Programming SkillsMax Woolf @minimaxir

Here is a common interview question that is typically gets asked in interviews:
"Please rate yourself in your best skill, on a scale of 1 to 10, where 5 is average."

The answers most often are between 7 and 9.5 inclusive, with an occasional 10.
Lets consider what that means.

Skills and ability to contribute have been shown to be distributed according to a Pareto Principle: 20% of the population possess 80% of the skills and deliver 80% of the results. Suppose, it is a Pareto distribution with the slope of 1, and a minimum skill value of 1. There is no max value for the true Pareto distribution, but the way developers are asked to use the scale, the max value is set at 10.

A few runs on University of Alabama in Huntsville Special Distribution Simulator show that for a 1000-point simulation the mean is in the 7-9 range. So far so good.

However, the randomly generated values have no set maximum, and can go well above 10. Even a few higher values in a large set skew the mean significantly.  To get back to the allowed range of values 1 through 10, replace all values in the 1000-point dataset, that are above maximum allowed value of 10, with 10.5. The average drops down to ~4.  The median is ~2, i.e. half the data points have values of less than 2.

That contradicts the initial question, which sets 5 as being the average.  A distribution of skills is likely to have an average level skewing lower than half the range. In this example of modeling skill distribution with a Pareto distribution 1,1 - way lower, at the mean value of 2.  More importantly, it raises the question on how well the interviewers understand the question they are asking the candidates.

"The best and the rest: Revising the norm of normality of individual performance."
by Ernesto'Boyle Jr., Herman Aguinis

All of the above is just a long way of saying that when a typical candidate for a position rates himself in response to the question, they are, so to speak, gaming the situation.

The more interesting observation is that not only the question is asked regularly, by many, if not most, developers acting in the interviewer capacity. The answer is taken seriously and can influence how the interview proceeds, as well as the outcome. People with higher self-rating (those claiming to be above 8) are more likely to be hired and promoted.

Honest or nice

We have a lot of conversations how we could do better. Deliver more functionality and prettier UI, cause fewer bugs, have more fun while we put in more hours.  Write better code, and pay back the technical debt.

At 200OK web professional's conference we were talking about code review – the part of software development process where developers and architects get together to consider other people’s code with the purpose of offering critique.

Having one’s code subject to review is terrifying for many people, and liberating for others.  It is also necessary, for most of the code outside of personal projects.  But how we are going about doing the review can make a huge difference for all involved.

There comes a scary idea of being nice to the people one works with. Genuinely, authentically nice – actually wish them to be successful, be willing to put effort in helping them, and talking about that.
Here are some suggestions:

• Start code reviews with saying to your fellow developers that the work they have done toward the team’s goal is noticed and appreciated.
• Call out good code – clearly expressed logic, fitting patterns, relevant abstractions, meaningful naming.   
• Notice good intentions, as expressed in code, even if the result is less than perfect. That includes error handling, code broken down into smaller modules, attempts at unit tests.  
• Finally, suggest changes as you would to a very senior, very experienced colleague – share knowledge while acknowledging their wisdom and understanding.  Everyone needs to learn, and everyone deserves to learn in a respectful, cooperative environment. 

Being actively and explicitly nice, yet honest, to one’s teammates does a few interesting things to overall team dynamic. More people speak up and offer ideas. Jerks become more visible. More conflict bubbles up and out, and leads to a healthy discussion.

It may even lead to delivering more value, while enjoying working together more.

Brown-field development

Image Copyright @ Robert Brooklyn http://unemployedprogrammer.blogspot.com/

Developers often complain about working on legacy code base. “Green field” development, i.e. writing brand-new systems where no code existed before, is relatively rare. Most software development work happens in the midst of the pre-existing and often relatively old coding.

Why is so much work being done on legacy projects? If people prefer to work on new code, and people in software development industry often get what they want – why isn’t there more of brand-new software projects?  After all, software developers nowadays get shiny offices with nap pods and gourmet catering, free laundry services and massages on-site.

It is well-documented that most of tech startups fail.  Most ideas do not lead to sufficient ROI and good business value. A lot of software projects get done [to some degree] and then thrown away, because they did not turn out to be viable enough to keep investing in. 

So what is left is a select few projects that turned out to be spectacular successes: profitable enough to keep using and getting business value from for a good long time. Organizations choose to continue to invest in these projects because older systems offer continuous income and a well-known ROI based on past history. While these projects are relatively rare, they tend to be large: small projects grow over time, and over 80% of investment in a software system happens after the project is declared done and enters maintenance mode.

Most of the non-startup development happens on the large, hairy, hugely successful projects with boring names and scrappy old interfaces. The code can be old, but the system is brilliant – it survived the competition with all other competing projects in its business space.  This is where the ROI and the business value are.  

The offices may be getting nicer and brighter, but we are likely to continue working on in the deep-brown for the foreseeable future.