Too big to fail

Have you been to Denver airport? It is one of the more modern airports in the US.  It handles vast numbers of travelers, with surges of visitors arriving for skiing weekends.  Many bring oversized ski gear, mountain bikes, various outdoors equipment.  The airport uses a manual system to handle the luggage. 

At the time Denver International Airport (DIA) was being built, there was a project to build a state-of-the-art automated luggage-handling system to improve speed and efficiency of the operations.  The luggage-handling system was being built by BAE Systems, an independent vendor who was originally hired for a smaller project.  When airport authorities sought bids for building the all-airport system, none of the submitted proposals promised to deliver this highly complex project within the offered timeframe.  So BAE was approached directly, and soon agreed to a fixed scope, schedule of under 2 years and fixed budget deal to deliver the all-airport baggage handling system.

As the project went on, it became apparent that the deal with BAE has been made in haste, many stakeholders in the system did not get a chance to provide their input.  Thus the project was pressured into accepting more changes later in the cycle.   Having started working on the system, BAE Systems ran into many technical problems both with system design and system constraints, while at the same time changes of directions were still being introduced.   In addition, death of Chief Engineer for the Denver International Airport and subsequent change of leadership complicated the project even further.

Throughout this ordeal, neither BAE Systems nor DIA team raised the question about terminating the project.  For nearly 3.5 years seasoned managers and engineers continued to press on (and spend taxpayers money) on a project that was late from the start, was handled by an inexperienced vendor, and was continuously pressured to agree to yet more last-minute changes, with no risk management or mitigation in place.

BAE Systems did not have the expertise in building systems of that magnitude, and was looking to gain experience on the Denver airport project so that it could pursue more big projects in the future.  While some people at BAE realized the risks and complexities, the management chose to press on because the stakes were so high.    DIA was shooting for a most modern, state-of-the-art airport facility, and deemed the project too big to fail. Both groups chose to ignore reality and expert advice in the pursuit of building the most complex system in the world.

DIA baggage-handling system of the 1990x is one of the more spectacular failures among IT projects to date.  Bad technical problems, poor planning, scheduling pressure, unexpected last-minute changes and complete lack of risk management all contributed to the situation.   The scary part is, all of these are present in most software projects.  High (or perceived high) stakes cause people to press on to the breaking point, demanding managers up the pressure, inexperienced technical people struggle with problems and plan poorly, over-confident customers insist on last-minute changes.   The successes still happen – often by a miracle of dumb luck, an amazing technical genius, blatant disregard to work-life balance, -- but not nearly as often as they could, and should.    

Style and substance

There are people who, when asked what car they would like to have, say "Red, with a sunroof and leather seats". A car is a powerful machine that exists to move people and/or things from one location to another, handle high speeds, protect its content from strong hits, and last a significant amount of time.   Leather seats are cool, but are not exactly high on my list of important characteristics of a car.

Leather seats are not unimportant.  They are one of the first things that a person notices about a car, and have a big impact on the driver's and passengers' experience.   However, car owner's experience is more than just driving and riding in that car: most car owners also have to survive an accident or a dozen while in the car, handle service and repairs for the car, and occasionally drive in adverse conditions where good handling becomes extremely important. 

If a car was a piece of software, the leather seats would be a feature.  Engine, handling and protection for the passengers provided by the strong body of the car would all be parts of the architecture.   Many people, when thinking about software, tend to consider its features, and nothing else.   When these people are product owners or project managers, the product features get a lot of thought and resources, while architecture evolves like a weed, with little care and hardly any testing.

Let me continue with the car analogy: an automobile can have the best-quality leather in the world on its seats and also be a complete lemon.  The wonderful seats won't make the car run, be safe or reliable. Same goes for software projects: great features do not guarantee usefulness, user satisfaction, or return on investment.  While features of an application make or break users' experience, the software, like a car, must also work, handle its load gracefully, and protect its owners and users from failures.   And that takes effort: time, resources, and careful thought about software architecture.

Ideas on software usability: power of a nudge

A nudge is a suggestion, a small incentive, but it can have big consequences on the outcome.

Placing certain products at or just above eye level in the shop is a nudge to consider these (presumably, most profitable) items first. Placing broccoli early in the lunch line-up leads people to get it more often, improving nutrition.  Automatic enrollment in 401k is a strong nudge to save for retirement, since the money will go in automatically.  Enrollment in a gym isn't much of a nudge, but signing up for a group exercise class or sessions with a trainer may work as a nudge and encourage attendance.

The nudge can come from a policy, from a physical environment, or from software.  Mozilla Thunderbird is smart enough to nudge the user to actually attach a file if it finds the word "attachment" in the text. Amazon.com and many other online retailers learned to nudge customers to write reviews, and to rate reviews left by others. Word processors nudge users to correct the spelling and sentence syntax by visually marking unknown letter sequences.

However, there is plenty of software that is less helpful than it could be if it was to nudge the user to follow better practices.

•  PowerPoint offers no nudges to produce good presentations - small font is as easy to reach as large, bullets go in indefinite layers with no suggestions to keep it simple. 
•  Modern IDEs have a lot of great capabilities to help write readable code, but don't yet automatically generate nudges to rename and refactor. 
•  Software like Infinitest nudges the developer to update unit tests regularly to keep them passing, but not to write new ones. 
•  Neither Wordpress nor Blogger.com nudge to limit the size of the blog post, suggest pagination, and do not remind to include pictures (but not too many pictures).

Behavioral research shows that nudges work. Yet, in order to work and improve performance, nudges need to be carefully designed and created. Software design industry is currently in the early stages of learning how.

A case for testing

Agile universe revolves around testing.  Developers are supposed to test, testers are supposed to test together with the developers.  Then the newly done work is supposed to be delivered to product owners and end users, who in turn get to try it, meaning, of course, to test it.

The result of this process is a high-quality product, where defects are known due to extensive testing, and features are exactly what the product owners and users have asked for and reviewed in their testing.    In fact, that’s the definition of a high-quality product – it does exactly what is expected of it, and its features are just what the doctor owner ordered, no more and no less.

However, there is a disconnect between the happy Agile universe, and the grim reality of software development industry. Testing is hard, expensive, decidedly not glamorous, and traditionally not valued very high.  For all the lip service that testing gets, there is relatively little research on the value of testing, how to do it right, how to measure the value, quality and quantity of the testing that does happen, and how much testing is enough. There is even less understanding of the importance of testing out in the field. 

Testing is a purely informational pursuit.  A test is something that’s done to learn how a product (a process, a person, etc.) behaves in a particular situation.  Modern software systems are very sensitive to lots of variables, i.e. “particular situation” is often extremely complicated to specify definitively.  Yet, this is what testing is all about – gathering information about what and how affects behavior of a system.

Regardless of how much work is done in testing, the reward and appreciation often goes not to the test and the people who performed it, but to the person or the team that changes (improves) the system behavior using the information gathered by testing. The idea that the improvement (or a bug fix) is wholly dependent on the information gathered in testing is not intuitive and usually gets lost in the shuffle.  A lot of information gathered by testing does not directly lead to a bug fix or improvement, and, thus, is not considered at all. 

It is hard to value the information, therefore, it is considered to be of low value.  It is easy to value the results of system improvements, therefore, they are valued highly.  As long as this mindset persists, the software industry will continue to under-test, and produce low-quality product that behaves unpredictably.

These thoughts are inspired by the recent news about Sony Playstation network compromised and subscribers' data possibly breached, private data of Texas employees handled unencrypted and left on publicly accessible website for months, Intuit's Quickbook service unavailable for many days earlier this spring.

Do you like to share?

The other day, I witnessed a professional instructor leading a group workshop refuse to let a person observe  the class without fully participating (read: paying). The workshop was focused on active student-teacher interaction, with hands-on work and individual attention being the key elements. A person quietly listening in was not getting anything remotely close to the full benefit of the class, and there was no additional burden on the teachers.  There was plenty of space in the room. Yet, that instructor felt that his work was somehow compromised by someone hearing what he had to offer for free.

It used to be the norm that people would guard their respective tools of the trade from colleagues, to prevent others from “stealing” their jobs. A database specialist would smile mysteriously when asked to please use SCM for his scripts, and a sales person would take her Rolodex home every night. Coders would get upset if others wanted to review their code, or ask permission to watch over their shoulder.

It seems that a lot of people in software industry have moved on to become more open about their work. We write blogs and produce podcasts to share hard-acquired knowledge, participate in question–answer forums to help others, and deliver talks at various user groups to all who cares to listen. Code reviews, while are not exactly industry standard just yet, are widely deployed in more progressive environments and are rarely considered disrespectful in and of themselves.

The perception of what is a skill has recently shifted from what a person owns – bits of knowledge and ideas – to what a person can do with these knowledge and ideas.  The skill is what enables a person to provide value, either by applying his knowledge and ideas to creating or improving a product directly, or by sharing with others.

As for the teacher who refused to let the person observe the workshop, he is well behind the times. And he lost a potential student that day.

---

AGILEDOTNET 2011 | Houston, TX – February 25th, 8am – 5pm | Register today www.agiledotnet.com