Estimates in the real world

Estimates sound like a simple, beautiful, thing no project should ever be without. As always, if something is too good to be true, it is not true. Having a detailed estimate up-front for something highly uncertain is desirable, and impossible.

Estimates are hard to use properly, and very hard to get value from. There are people who understand probabilities, have a good grasp of variability and resulting deviations. These people are able to work with, and to benefit from, early estimates while considering all the uncertainty of the situation. These people also do not exist in IT project-planning roles. 

Estimates look like a wonderful thing to have, but they are hardly necessary. Not having a good estimate may be less than ideal, but never disastrous. Estimates do not change the flow of work to the better, although they could encourage cutting corners.  Many projects have been successfully completed without any estimates, or with bad ones. 

Overall, the most desirable estimates are impossible to get right, all estimates are easy to misuse, and even the best estimates provide limited value even if everything goes well. Estimates are normally discussed in terms of a best-case scenario, which is very different from what usually happens.

• Best case scenario
• On-point estimate. All necessary resources available when needed. No changes in requirements.
• On-time and on-budget delivery.
• What usually happens
• Best-case scenario estimate, further shortened because of business need.  All necessary resources are not available until much later, except a few, that are not available at all. Massive and conflicting changes in requirements. 
• Complete chaos when estimated delivery date arrives.

Estimates are a nice thing to have in a perfect world.  Having good estimates allows to predict far ahead of time when and what resources will be needed, when and what milestones should be completed, and when to expect the final delivery. However, in a perfect world, resources are available quickly, milestones are hit reliably, and delivery has perfect timing - whether there were estimates or not. 

Real world is far from perfect. In the real world, more often than not, estimates are way off, inputs are late, milestones keep changing, and final delivery turns out to be a first draft.  The only piece that remains set in stone is the original estimate, while the rest of the situation changes dramatically. Rough guesses become deadlines, do-or-die commitments, and the cornerstone of blame-passing culture. 

For those projects that attempt to succeed in the real,  rather than perfect, world, consider doing away with the estimates-come-deadlines.

For hard projects, choose Agile

What’s the cheapest way to build software?  

Agile approach is expensive.  There are meetings that take valuable engineering time. There are non-engineering roles, i.e. people that need to be paid, that are not writing code and, therefore, whose contribution to the product comes into question.  There is a significant body of test code, and which is obviously not helpful to the end user. There is constant refactoring work as the team discovers better architecture for the emerging product.  

Waterfall is cheaper. Only hire an architect to for the initial phase of designing the system.  Pay the technical team just to write the code for user functionality, and to do it quickly.  The engineers can be cheaper, too, since all the thinking has already been done in the earlier design phase.  Let the engineers go once all the code is written. Next QA comes in, runs tests, and violá, the product is done.  No meetings ever needed, since all the people involved are working off the earlier phase’ documentation.

Why pay more?

Agile process has no single point of failure. Fault-tolerance of the process is extremely high, error detection and recovery is “baked in”. Waterfall, in contrast, requires perfect execution at every point in the process, i.e. every point is a single point of failure. Errors are hard to detect until the very end, and even harder to fix. As projects get more complex and require more investment, it makes sense to place a higher value on the probability of success, rather than on minimizing initial cost. 

"Slowly and carefully"

I once asked a coworker with an official title of Senior Software Architect, how he makes decisions and gives recommendations on the architecture of our product. “Slowly and carefully”, he replied. He was an honest and hard-working person, a very smart guy, and probably a decent architect. Yet our product’s architecture was terrible, and it was not improving as time went by.

Two deadly sins of software architecture were ubiquitous in our code base

• Everything was tightly coupled with everything else
• Same information was defined in multiple places

In day-to-day work these problems caused lots of hard-to-trace bugs, showing up in seemingly unrelated portions of the code. Changes required touching lots of code, leading to more bugs. And it was very hard to write tests – so tests were not written.

So, what was my colleague doing wrong? The architectural changes he and the team inflicted on the code base were fine - mostly because very little changes were made. There was no opportunity for a serious refactoring effort to improve on the underlying structure. The bad stuff that accumulated over time continued to accumulate. The new features represented good ideas, but were poorly implemented – or, rather, were implemented in much the same way as old ones, creating more bad code.

Looking back, there was a culture problem with the approach of “slowly and carefully, with no objective criteria, teamwork, or testing”. The architect did not write code. He provided suggestions that were implemented by other people. Programmers wrote code per instructions, and did not create tests to fully exercise the added or modified code. The architect did not have the feedback on how well his ideas fit into the code base, and programmers had very vague feedback on how their changes affected the system.

Most of the time, the architect's solutions were fine.  He was familiar with the code, has been on the project for a long time, and knew the team well. But since his suggestions were hardly ever challenged, and never tested against competitive ideas, it is impossible to tell whether what he offered was really good - or barely good enough.

Every so often, a developer would complain that the solution suggested by the architect did not work, and it became a point of pride for others to write some “clever” code to fit the square peg (architect’s suggestion) into a round hole (existing code). It is well established that the “clever” code is more bug-prone and harder to maintain, compared to straight-forward and easily readable code.

Sometimes, no amount of cleverness could fit a new elegant idea into the clunky existing code, and then people improvised on their own. Most of the time, those were good, solid solutions, but not always. There also was an illicit feel to the situation, and some developers tried to avoid code reviews in these cases. There were a few scary monstrosities quietly sneaked into the code base as a result.

Splitting the implementation and design work prevents everybody from learning from their successes and mistakes. Having an architect dictate unchallenged solutions creates sub-par performance, tension and funny feelings even on the best-intentioned team.