Myths and half-truths about estimates

Estimating is in our DNA. It is about the future, the knowledge, the power – all the things that make us feel in control.  Estimates are required, and are the most straightforward (or even the only) way to gain trust, and budget, to go forward.

Yet… it is an open secret that estimates are flawed. Things do not go as planned. Resources become unavailable, assumptions turn out to be incorrect, changes trickle down in a never-ending stream, and communication becomes strained.

While having flawed estimates may be preferable in some situations to having none, commitment to wrong numbers can wreck havoc on any project. If you are going to estimate, steer clear of these mistakes people frequently make, even if they have plenty of experience with estimating.

• Myth: If your estimates are a range of dates, you are doing a good job managing expectations.
• Only the earlier, lower, more magical numbers will be remembered. And those will be accepted as firm commitments.
• The lower bound is usually set at "the earliest date the project can possibly be completed". In other words, there is absolutely no way the work can be completed any earlier, even by a day. What are the chances of hitting that exact date? Practice shows - close to nil. 

• Half-truth: You can control the quality of your estimate by putting more work into producing this estimate.
• By spending some time learning about the project, researching resources available, considering major and minor risks, one can produce a somewhat better estimate.
• The above activities are only going to take the quality of the estimate so far.  Past a certain point, no matter how much effort goes into estimating a project, the quality of the estimate is not going to improve. Then the best bet is to simply start working on the project.

• Myth: People can learn to estimate better, as they gain experience.
• It is possible to get better at estimating – if one keeps estimating the same task, which becomes known and familiar with experience. This is hardly ever the case in software development. Every project is different, most teams are brand new, and technology is moving along fast enough.
• Do not expect to produce a better estimate for your next project than you did for your last one.
•  By the same token, do not expect a worse estimate. The quality of the estimate is going to be low, and it is going to be random.

• Half-truth: it is possible to control the schedule by giving up quality.
• Only for short-term, throw-away temporary projects.
• For most projects, aiming for lower quality has a negative effect on the schedule.

The human side of enterprise

Once upon a conference, I asked a small technology business owner about the interactions his 12-person team had, and he was sincerely surprised by the question. “Why would they need to talk? I tell them what to do, and then they code, each in their cubicle.” – was his response.

We like to think about work strictly as job function-dictated activities. Gathering requirements, evaluating options, making product decisions. Meeting to discuss who does what, how hand-offs happen, what are the acceptance criteria. Org chart-based relations that dictate who is in charge, who reports to whom, and who will get the credit or blame. The result of these interactions is work product, the how-to of getting to a solution.

Yet, there is more to work than “just work”. It is also the true, human relationships we develop with people we spend a lot of time with most days of the week. A quick exchange by the water cooler about a concert, a site of a geeky new mouse pad on someone’s desk, kids’ pictures on the screen. A small group is discussing the latest smartphone while getting coffee; when waiting for a meeting to start someone mentions that she is looking for a rock-climbing gym; and somebody else is asking for advice on the user groups during lunch.  The result of these conversations are not necessarily a work product (although it could be directly useful for work, too), but the building of a team of friends.

 

A team comes together through learning small, personal, often seemingly insignificant details about each other, discussing experiences, finding shared interests. Working with friends matters. It is a lot easier to be open-minded and welcoming to a friend’s ideas, than to someone’s who is a stranger. Friends deal with disagreements by working together, while strangers get upset and push each other away. Knowing and liking people one works with makes everyone happier and more productive.

When the opportunities for social interactions within a team are limited, work suffers. Teams of strangers are less interested in working together cooperatively, sharing information, establishing and striving toward a common goal. There is more shirking, less innovation and leadership. And less success.

Simple solutions to complex problems

Complexity and simplicity are often, and intuitively,
regarded as two extremes of the same continuum,
or spectrum. Yet, this may be a simplistic view, indeed.

Complexity of a solution reflects many things: complexity of the problem, of the problem domain, the level of understanding of the above by the people creating the solution, and the complexity of the organization and communication of the people in charge of building the solution.  Finally, complexity of the solution will reflect the incentives people were facing: was it in their interest to create a complex solution? Or to make the solution as simple as possible?

Integer calculator is inherently less complex than, say, mapping software, but implementation of the algebra functions can be ridiculously complicated. Or fairly simple, depending on the attitudes of individual architects and implementers, and their organization.  Business rules expressed as decision trees are relatively simple, yet there are plenty of solutions in that space that are mind-bogglingly complicated.

Agile approach results, among many other things, in reduced complexity in the delivered product. Cross-functional self-managed teams, while not easy to build, are a lot less complex than matrix organizations of the traditional firms.  Communication patterns are simpler and more straightforward. Preferring a team of generalists over narrowly specialized experts levels out the playing field, and encourages more level understanding of the problem domain within the team. The emphasis on speedy delivery of small slices of the solution leads to the team concentrating on the “lowest hanging fruit” at any point in project timeframe, ensuring that the team works on the best-understood pieces of the problem, while it expands its understanding.

Engineering practices that are often coupled with Agile also carry a significant focus on simplicity in design and implementation. Emerging design, developed with continued and immediate feedback from the implementation and user assessment, is simpler than the up-front design created early in the project, prior to having project work feedback loop established. Unit-level automated tests are simple to create and maintain, and having adequate test coverage and infrastructure reduces both probability and complexity of inevitable bugs.

Frequent collaboration with users and customers, required by most Agile frameworks, forces all parties involved in a project to develop a joint understanding that eventually becomes the basis of the solution. Joint understanding of a diverse group is always less complex and less nuanced than that of a single mind (participating in the group), or a similarly-minded group, thus leading to a simpler solution.

As the business of software development moves forward, it is taking on more complicated problems, and attempts to deliver increasingly complex solutions. Keep the complexity in check by simplifying communication paths, promoting learning and feedback, and striving for joint understanding among diverse groups.