Share it

Thursday, July 17, 2014

Empowering Agile teams: culture clash

"The test of a first-rate intelligence is the ability 
to hold two opposing ideas in mind at the same
 time and still retain the ability to function." 
- F. Scott Fitzgerald

Agile frameworks empathize empowered teams, flat structures, and meritocracy. Team members and those in supporting roles are expected to earn trust and respect, rather than get authority based on org chart.

Organizations in general do not work this way. A modern organization is all about hierarchy, where authority comes from one’s position and place on the chart. Responsibility and decisions each travel in exactly one direction in the hierarchy: decisions come down, responsibility goes up.  This kind of structure is easy to manage from the top. But it does not lead to best decisions.

An Agile team is encouraged to take control of, and responsibility for, their own work, but it comes in direct conflict with the not-so-Agile organizational structure. Entire corporate cultures are built around deferring to someone with a higher title. Those cultures do not disappear when an authority figure up the hierarchy decides that some projects should try Agile.

People on an Agile team are taught to care about producing value, make their own decisions, and take responsibility – but for employees with years of experience in traditional organizations it is not easy to accept this kind of message at face value.  It is even harder if Agile has a narrow mandate, for example, when transformation is happening only in a small part of organization.

This cultural shift is the hardest and the most painful part of a large-company Agile transformation.  It is also the most valuable change a business can make to improve its culture.

Incredibles 2006

Everybody wants a pony


In a recent conversation, a colleague mentioned that a project his team worked on was considered a failure. That surprised me. I heard that this team delivered a high-quality product; the software quickly went into production, and has been delighting users for some time now. 
Turns out, when the team first got started on a project, it came with a specification document and a deadline. As the work proceeded, the team was learning more about the project. A number of features were added based on conversations with stakeholders and organizational knowledge, other features described in the specification document were cut. Technical complexity was uncovered, and eventually conquered.  The team iteratively delivered the system for stakeholder review, each time presenting more functionality, and at some point it was decided that the software is ready to go into production. 

Well, as it happened, for this particular project the production release date was after the original deadline set on the project, back when the project consisted only of a specification document and that deadline. Therefore, the deadline was declared missed, and the project was declared a failure. 

I find this mind-boggling.  Before anything was known about a project except that there is software to be built, there was this random date.  Now, that the world moved forward, the project came into existence as a working, helpful system that delights users and produces real, tangible value for the organization. Yet, somehow, that original date is still the deciding factor whether the project succeeded.  Conforming to this picked-before-we-knew-anything-important date is more important than whether the project was delivered at all, whether it useful, or valuable, or makes users happy and productive.

It's understandable that people and businesses want a pony – projects that are predictable enough to be able to forecast completion date, scope, and user satisfaction ahead of time.  In practice, most of software development work happens on complex projects, with a lot of unknowns, where information is uncovered gradually as more work is done.  There is a very little chance that any given project is a pony. More likely it is a zebra, with a variety of stripes, and lots of unpredictability.  


Saturday, July 5, 2014

Estimates in the real world


Estimates sound like a simple, beautiful, thing no project should ever be without. As always, if something is too good to be true, it is not true. Having a detailed estimate up-front for something highly uncertain is desirable, and impossible.

Estimates are hard to use properly, and very hard to get value from. There are people who understand probabilities, have a good grasp of variability and resulting deviations. These people are able to work with, and to benefit from, early estimates while considering all the uncertainty of the situation. These people also do not exist in IT project-planning roles. 

Estimates look like a wonderful thing to have, but they are hardly necessary. Not having a good estimate may be less than ideal, but never disastrous. Estimates do not change the flow of work to the better, although they could encourage cutting corners.  Many projects have been successfully completed without any estimates, or with bad ones. 

Overall, the most desirable estimates are impossible to get right, all estimates are easy to misuse, and even the best estimates provide limited value even if everything goes well. Estimates are normally discussed in terms of a best-case scenario, which is very different from what usually happens.
  • Best case scenario
    • On-point estimate. All necessary resources available when needed. No changes in requirements.
    • On-time and on-budget delivery.
  • What usually happens
    • Best-case scenario estimate, further shortened because of business need.  All necessary resources are not available until much later, except a few, that are not available at all. Massive and conflicting changes in requirements. 
    • Complete chaos when estimated delivery date arrives.
Estimates are a nice thing to have in a perfect world.  Having good estimates allows to predict far ahead of time when and what resources will be needed, when and what milestones should be completed, and when to expect the final delivery. However, in a perfect world, resources are available quickly, milestones are hit reliably, and delivery has perfect timing - whether there were estimates or not. 

Real world is far from perfect. In the real world, more often than not, estimates are way off, inputs are late, milestones keep changing, and final delivery turns out to be a first draft.  The only piece that remains set in stone is the original estimate, while the rest of the situation changes dramatically. Rough guesses become deadlines, do-or-die commitments, and the cornerstone of blame-passing culture. 

For those projects that attempt to succeed in the real,  rather than perfect, world, consider doing away with the estimates-come-deadlines.