Estimating Sprints aka the problem with Scrum

Agile landscape assumes lots of quick churn  - requirements grow and shrink, product scope shifts, technologies are tried on for fit, even delivery platforms seem to be switching multiple times within project life cycle.

The time where the team has had a chance to learn enough about the tech, the requirements, the platform, etc. appears to not come until much later in the project. In the meantime, while the team figures out the tech and goes along with the experimentation of what the users will find most useful, every estimate is a wild guess.

Yet, every two (or one, or three) weeks, there’s the big Question: what do we pull into this Sprint?
On one hand, there are learnings from past Sprints – some things have gotten easier, other things were dropped because they could not be done, some architectural decisions have been made. One the other hand, past Sprint experience shows that unexpected things invariably come up and slow progress to a crawl.

Then there is reporting. Forward-looking reporting means it looks bad if the team has not completed everything it has pulled into each Sprint.  From this perspective, it is better to estimate conservatively.  Backward-counting from a deadline where currently-uncertain features are assumed to be required, and therefore must be packed into prior Sprints with no room to spare for dealing with the unexpected.  There is very real pressure to load up the Sprint with work that could possibly be all be delivered only under the best possible scenario.

A less-than-pure approach is flexibility: plan the Sprint conservatively, with the understanding that the Scrum Master together with Product Owner will prioritize backlog to pull work into the Sprint if there is time remaining.  This is less than ideal because having more on the plate leads to higher productivity overall – even if it’s not possible to complete everything.  Alternatively, a team can choose to plan the Spring aggressively, pulling more work in, and then reducing the Sprint load if necessary.   Unfortunately, this looks really bad from the outside.  Well-meaning teams can be penalized for under-delivering relative to an aggressive Sprint forecast - even when that Sprint forecast was deliberately set up to be unrealistic. 

Another option to deal with the planning question is to go back to the world where teams had a lot of scope-platform-design decisions made up-front, and could provide somewhat reasonable estimates 3-4 Sprints into the project.  While those estimates weren’t exactly right, there was predictability earlier in the project cycle.  But that would mean giving up the opportunity to learn and to benefit from new discover as the project is being worked. 

[ This post is a first part of a series. To be continued. ]

Achieving more by blaming less

Complex, consequential and time-sensitive problem-solving is best done in the atmosphere of full transparency. Alleviating fear of blame and sanctions is essential for having an honest conversation, taking social and technical risk, and otherwise focusing on doing valuable and creative work, rather than avoiding negative consequences. Not fearing consequences unleashes creativity and striving for better results, even if it involves risk, is correlated with great success in high-tech.

Sanction-less and blameless are both good, but not identical ideas.

Sanction-less means that those decided to be at fault are not going to be sanctioned – written up, demoted, fired, or otherwise punished. Sanction-less must come from the very top of the organization to be trustworthy, and is fairly clear-cut.  The promise of “no punishment” can still be broken at any time, but in many organization past history tends to be a reasonably good indicator of the future behavior.  If the management promises “sanction-less”, and has not levied sanctions for mistakes leading to serious and costly incidents in the past, it is likely that the next incident investigation will adhere to the same policy.

Blameless is more complicated.

Blame is a social construct, and involves interpersonal relationships, rather than company policy. Blame can be levied by any person involved in the situation, from the team members to stakeholders to users to bystanders. While blame can be accepted or not by the person who is being blamed, the blame is created by the person or people doing the blaming. It is easy to create blame – all it takes is one person, with any title, role or level of involvement. Once created, blame does not go away. It can slowly fade with time, it can also turn around on the people doing the blaming if it becomes known that the blame was created in error.  Blame often continues to affect relationships and interactions for a long time, even the relationships that were created after the blame first came into existence.

Blame is a natural human reaction in response to a negative situation.  While the goal of dealing with an incident is to recover and, if possible, to make up for lost value, people typically want to know what caused the problem, and who is at fault.  With wanting to know who is at fault comes the propensity to blame the negativity of the event on that person or people.

Is it possible to get away from the blame, and to create a blameless culture?

There are a couple of ways.  One is explicit and relentless focus on group responsibility, so that personal responsibility is forcibly completely disregarded. While actions are taken by specific people, it is possible to gather all accountability at the team level.  As causes of the problem are discovered, ignore the name or names attached to the action that created them to the best of your ability, to avoid blaming these people.  

The other way to avoid blame is to focus on the future rather than the past.  Do not look for the cause of the problem. Instead, look for a way to solve the problem going forward.  Work from the solution, rather than the cause, to make changes needed to avoid the same or similar problems in the future. The benefit of this approach is that all the effort goes into resolving the existing situation, rather than discovering and evaluating the past.

Blameless culture is not easy. Problems can be severe, emotions can run high, and searching for cause is deeply ingrained into our understanding of how work should get done. In addition, our tools are great at keeping logs on who did what and when.  

Knowing who caused the problem is synonymous with assigning blame. It is human nature to judge, to evaluate, to critique, and, yes, to blame. It is also human nature to guard against blame-inviting behavior, and not only we try to do well, we also try hard to hide what we did less well.  A lot of effort and cognitive strain goes into avoiding and escaping blame, however futile. 

Achieving blameless culture allows to avoid this waste, and more fully focus on other goals. Consciously working toward minimizing blame promotes healthier relationships, better emotional experience overall and specifically while dealing with severe incidents, and encourages more honesty and transparency.

Achieving success

When considering performance, there is typically more focus on failure than there is on success. Failures are often noticeable, well-remembered one-time events that generate high emotions and can affect many people.  There is the idea that teams learn from our mistakes and failures, and that in order to improve teams must fail publicly and painfully. It is believed that significant successes must somehow be rooted in overcoming past misfortunes and failures.

Success can be loud and come with a lot of fanfare as well, but the fanfare of success tends to fade faster.  New challenges pop up and emotions go away quickly because the team must get back to work, as success tends to bring in more work and generate additional challenges.  

As a result, success appears to play a smaller role in overall evaluation than failure.  That is both unfortunate and counter-productive.  

It is unfortunate, because failures are not necessarily what makes us learn and improve going forward. While occasionally there is learning that is an outcome of failure, the same learning can also happen without experiencing the fail.  A lot of failures are unavoidable, but nevertheless result in blame (or self-blame) and reduce motivation.

Focus on failures is also counter-productive, because failure is hardly ever the enemy. Failures come from experiments, from aspiring to reach farther, from ambition.  Statistically, it takes a number of failures to achieve significant success.  Without these failures, the success is either impossible, or a lot less likely.

Success is not the opposite of failure, rather, it is a significantly different happening.  Success often comes with more work and more responsibility, creates additional challenges, and pushes the team to work harder.  Success requires active learning and deliberate practice, and there is a need to learn still more to handle challenges brought on by the achievement.  Success encourages ambition and experimenting, and boosts motivation.

Success is the main driver of future success. It is time to focus more on success, and less on failure, when considering team performance. 

Deciding the architecture

Agile project starts by gathering an Agile team, deciding on a project and specific deliverables to be built first, and setting up the needed infrastructure. 

But before the team goes about creating the infrastructure, several questions about the system architecture need to be addressed.  What infrastructure will be needed, and what skills the team must have, greatly depends on the proposed system architecture and technology stack.

Who decides system architecture is the “elephant in the room” issue of Agile projects.  Many smaller design decisions will happily “emerge” as the team goes through the daily work of building the project iteration after iteration. Still, there is a number of large-scale, long-lasting choices about system architecture, that must be decided at the start of the project, when relatively little is known and the team has the least amount of experience on the project.

Who should be making these decisions? The typical answer appears to be The Architect, someone high up in the ivory tower, someone who makes no mistakes by producing little to none tangible, and thus imperfect, results. Sometimes that person is very good, and in other cases they just get by.  As it turns out, making the very best initial architectural decisions for a given project is not particularly important for the project's success, and making passable architectural decisions is fairly easy and fool-proof. 

List of common architectural patterns is very small, and almost all projects fit nicely into that tiny set.  Even if an inappropriate pattern is picked, the team will be fighting, and winning, over the inappropriate pattern to create a working implementation.  If the system becomes successful despite poorly designed architecture, and fighting the system setup becomes a serious obstacle to evolving the product, it will get re-written with a less inappropriate architecture shortly.

P.S. 

Sydney Opera House is one of the most beautiful and distinctive buildings in the world. Its architecture went through a dozen iterations of design, and the finished structure exhibits terrible acoustics - a big problem for an opera house and performance venue.  The building took 10 years longer than planned, and went 1,357% over budget. Of course, the project was, and still is, a resounding success.

Scrum as a series of feedback cycles

• Daily scrum: every 24 hours, within the Scrum team. Feedback on technical details.

• Sprint review: every 1 to 3 weeks, within the product team. Feedback on requirements, features delivered, and implementation details.

• Retro: every 1 to 3 weeks, or, better yet, on as-needed basis, within the Scrum team. Feedback on communication, processes, resources.

• Backlog grooming: every 1-2 days or even more often, within the product team. Feedback on features delivered, technical details communicated, and user feedback.

• Sprint planning: not a feedback cycle on anything… why is it a part of Scrum?

A key to the kingdom

Agile talks about emergent architecture and building vertical slices of functionality. It is typically understood that all efforts must go toward creating valuable user-facing pieces, as opposed to delivering software layers that cannot be used until all other layers are in place.  Unfortunately, this approach is often taken as “all we need is the visible parts”, i.e. the parts that the user gets to see and click on, or that push the data to be displayed.

When I first encountered a large DB supporting a well-used, living application with no indexes or foreign keys, I thought it was a rare oversight.  Then I encountered a few more database setups just like it, with large and heavily used tables and relationships, but neither indexes nor keys defined.  In all cases, users were complaining about slow response times, and administrators remembered more than one instance of the application becoming non-responsive under heavy load as DB requests timed out en masse.

Other developers I talked to also mentioned that they worked with reasonably large DBs that were missing indexes and foreign keys. A number of people, working in different companies on unrelated projects, mentioned that their engineering managers or software architects absolutely refused all suggestions to create even primary keys on tables “since nobody sees them anyway”. Several developers talked about applications being re-implemented, in part, because of poor response times caused by frequent timeouts of DB requests.

Vertical slices of user-visible functionality must include “the plumbing” that allows the application to handle the expected load. “Emergent architecture” is not complete without at least basic considerations of technical quality. A project cannot succeed without understanding what it takes to deliver user value in a production situation – and that typically includes multiple users working simultaneously.  Building in proper DB infrastructure is a simple example of such considerations, the work that is not visible but nevertheless valuable to the user.

When the math is off

How Developers Rate Their Own Programming SkillsMax Woolf @minimaxir

Here is a common interview question that is typically gets asked in interviews:
"Please rate yourself in your best skill, on a scale of 1 to 10, where 5 is average."

The answers most often are between 7 and 9.5 inclusive, with an occasional 10.
Lets consider what that means.

Skills and ability to contribute have been shown to be distributed according to a Pareto Principle: 20% of the population possess 80% of the skills and deliver 80% of the results. Suppose, it is a Pareto distribution with the slope of 1, and a minimum skill value of 1. There is no max value for the true Pareto distribution, but the way developers are asked to use the scale, the max value is set at 10.

A few runs on University of Alabama in Huntsville Special Distribution Simulator show that for a 1000-point simulation the mean is in the 7-9 range. So far so good.

However, the randomly generated values have no set maximum, and can go well above 10. Even a few higher values in a large set skew the mean significantly.  To get back to the allowed range of values 1 through 10, replace all values in the 1000-point dataset, that are above maximum allowed value of 10, with 10.5. The average drops down to ~4.  The median is ~2, i.e. half the data points have values of less than 2.

That contradicts the initial question, which sets 5 as being the average.  A distribution of skills is likely to have an average level skewing lower than half the range. In this example of modeling skill distribution with a Pareto distribution 1,1 - way lower, at the mean value of 2.  More importantly, it raises the question on how well the interviewers understand the question they are asking the candidates.

"The best and the rest: Revising the norm of normality of individual performance."
by Ernesto'Boyle Jr., Herman Aguinis

All of the above is just a long way of saying that when a typical candidate for a position rates himself in response to the question, they are, so to speak, gaming the situation.

The more interesting observation is that not only the question is asked regularly, by many, if not most, developers acting in the interviewer capacity. The answer is taken seriously and can influence how the interview proceeds, as well as the outcome. People with higher self-rating (those claiming to be above 8) are more likely to be hired and promoted.

Honest or nice

We have a lot of conversations how we could do better. Deliver more functionality and prettier UI, cause fewer bugs, have more fun while we put in more hours.  Write better code, and pay back the technical debt.

At 200OK web professional's conference we were talking about code review – the part of software development process where developers and architects get together to consider other people’s code with the purpose of offering critique.

Having one’s code subject to review is terrifying for many people, and liberating for others.  It is also necessary, for most of the code outside of personal projects.  But how we are going about doing the review can make a huge difference for all involved.

There comes a scary idea of being nice to the people one works with. Genuinely, authentically nice – actually wish them to be successful, be willing to put effort in helping them, and talking about that.
Here are some suggestions:

• Start code reviews with saying to your fellow developers that the work they have done toward the team’s goal is noticed and appreciated.
• Call out good code – clearly expressed logic, fitting patterns, relevant abstractions, meaningful naming.   
• Notice good intentions, as expressed in code, even if the result is less than perfect. That includes error handling, code broken down into smaller modules, attempts at unit tests.  
• Finally, suggest changes as you would to a very senior, very experienced colleague – share knowledge while acknowledging their wisdom and understanding.  Everyone needs to learn, and everyone deserves to learn in a respectful, cooperative environment. 

Being actively and explicitly nice, yet honest, to one’s teammates does a few interesting things to overall team dynamic. More people speak up and offer ideas. Jerks become more visible. More conflict bubbles up and out, and leads to a healthy discussion.

It may even lead to delivering more value, while enjoying working together more.

Brown-field development

Image Copyright @ Robert Brooklyn http://unemployedprogrammer.blogspot.com/

Developers often complain about working on legacy code base. “Green field” development, i.e. writing brand-new systems where no code existed before, is relatively rare. Most software development work happens in the midst of the pre-existing and often relatively old coding.

Why is so much work being done on legacy projects? If people prefer to work on new code, and people in software development industry often get what they want – why isn’t there more of brand-new software projects?  After all, software developers nowadays get shiny offices with nap pods and gourmet catering, free laundry services and massages on-site.

It is well-documented that most of tech startups fail.  Most ideas do not lead to sufficient ROI and good business value. A lot of software projects get done [to some degree] and then thrown away, because they did not turn out to be viable enough to keep investing in. 

So what is left is a select few projects that turned out to be spectacular successes: profitable enough to keep using and getting business value from for a good long time. Organizations choose to continue to invest in these projects because older systems offer continuous income and a well-known ROI based on past history. While these projects are relatively rare, they tend to be large: small projects grow over time, and over 80% of investment in a software system happens after the project is declared done and enters maintenance mode.

Most of the non-startup development happens on the large, hairy, hugely successful projects with boring names and scrappy old interfaces. The code can be old, but the system is brilliant – it survived the competition with all other competing projects in its business space.  This is where the ROI and the business value are.  

The offices may be getting nicer and brighter, but we are likely to continue working on in the deep-brown for the foreseeable future. 

The bug of security theater

It seems like the Internet and web-based services have been around forever and work well – from anywhere, just open up a browser on any device that supports Internet browsers. Well, at the end of 2016, there is a new bug going around, preventing many different services from serving. The bug of security theater.

After many a security breach, billions of records exposed or stolen, millions of people affected in some serious or minuscule or unknown ways, the providers have learned that security matters. However, security breaches are still relatively rare, unsystematic, and often caused by an ‘oops’ – some singular event that wasn’t supposed to happen, like operator error.  In many cases, there is simply not enough information how to prevent the break-ins and unwanted exposure or loss of data. In other cases, providers may be able to do somewhat better – but at the great expense of re-training staff, updating and enforcing stricter policies, and re-working technical systems.  

Still, there is something that is relatively cheap and easy to do, and improves security – at least in the eyes of users and media.  Security theater is commonly show-cased in the user interface. Many-factor authentication, secure codes sent by email or text message or by audio in a phone call, highly sophisticated personal and public questions going back many years for the users to answer – all for the privilege of accessing the same old web-based email account.

Many providers are requesting a smart phone# to authenticate against, which is good for them – more data, but bad for us – more advertising phone calls.  Many providers require a live exchange of data over the phone or email before allowing access, which is hardly good for them, and definitely bad for us – the wait times are often unreasonable.  Still others insist on following up online communication with a phone call – which is bad for everybody, because if we wanted to communicate by phone, we would not bother with online access at all.

Security is different from security theater.  Security is hard, and should enable us to do more business. Security theater is cheap, generates false sense of being protected (and very real frustrations), and prevents us from doing things we want to do. 

Conversations on diversity

I want to share a few  interesting conversations with or about women in technology industry: 

A young white American male, on attending Grace Hopper Celebration - a conference by and for women in computing

- It felt really weird to be part of the small minority of men there. I usually feel like I belong when I go to technology events, but this was different. 

---

A different young white American male, works in a heavily male-dominated office

- So, as a member of the majority, what can I do to welcome more women to participate in tech?

Another young white American male, same workplace

- But we welcome women! We are a total meritocracy.  Women just do not apply.

---

Mid-career software professional, female, working in the public sector

- Yesterday, I suggested researching a way to automatically add new users to "AR" group. I told it three times. It was ignored and dismissed. By the end of the meeting, Garry said exact same thing. Adolfo carefully recorded it in his plan of action.

Experienced white male, in response to the woman’s comment above

- Sad that it upset you so much. How much easier would be your work if you just shrug and ignore, even better, do not notice these little injustices. The credit, that little bit of authorship honor, was stolen from you. The whole incident was a waste of time and emotions.

---

Another professional woman

- I get a flood of emotion when I realize that consistently in our staff meetings one of my male coworkers echoes my comments or objections loudly for the whole group, either in agreement or augmenting them with his own thoughts in legitimate discussion. This is a first in my career that I feel consistently heard, even with the many female coworkers I've had. And he is raising a daughter.

Brilliant hare?

We called him Tortoise because he taught us. - Lewis Carroll

In a conversation about work, a colleague mentioned that a person who was perceived as absolutely brilliant by everybody who knew him. People believed he was brilliant, because he was always coming up with answers for every complicated technical problem or question very quickly.

Being quick is important for success. ‘Quickly’ is one of the more popular words in LinkedIn recommendations. Speed-reading, speed-listening, and occasionally even fast typing, are considered good and important skills for technology workers.

Yet, quick can be an enemy of good. Quickly pushing out code typically leads to bugs, technical debt, and poor architecture. Quick decisions often turn out to be poor – or maybe not, but only if they happened to be lucky, rather than well thought through. The engineer everyone’s raving about for quickly fixing bugs, is quietly introducing dozens of new ones.

“Thinking, Fast and Slow” by Daniel Kahneman talks about the different ways people are wired to think, one by applying simple rules and pre-existing biases, another for consciously considering all available information. The ‘quick thinking’ leads to stereotyping, emotional, and subconscious choices. The slow option is the opposite – calculating, conscious and effortful.

It may be time to rethink what it takes to be brilliant. Slowly. 

Work hard, aim high, fall, and keep going

I was told that this young woman in a sparkly dress is a top-level athlete at a local ice skating rink.  She came out on the ice with powerful confidence of someone who’s done that a hundred times. Indeed, she was very good - graceful glides, balanced turns, fast spins. Then she jumped. And fell. Got up and kept skating, and then jumped again. And fell again.

Those were complicated, high above the ice, many-turns jumps, and she did manage to land nicely a few of them. Still, she fell four times in a two-minute skating routine, all the way down on the ice. And each time she got up, picked up the beat in the music, and continued skating the program. More than that, she continued to attempt those high complicated jumps over and over again, never mind the falls.

The crowd was watching from the bleachers in awe. Down on the ice, this young woman was living through these, most likely unremarkable, minutes of her life – work hard, aim high, fall, and keep going at it. Throughout the evening, the skater kept attempting more jumps, sometimes landed nicely, but quite often continued to fall. She was the best, and that was exactly how she got to be the best, by pushing through the failure, the pain, the embarrassment, no matter the falls, jump after jump. 

Legacy code cycle

“Remember, code is your house,
and you have to live in it.”
― Michael C. Feathers
"Working Effectively with Legacy Code"

Every project goes through stages. First, there is excitement of design, green-field development, seeing things work. Then there are bug fixes, additional features that may or may not fit the originally designed architecture, little tweaks and changes to accommodate the cases not covered in initial development. And then there is support – making changes based on the usage patterns, keeping up with changing environment, fixing the harder-to-find bugs.

Successful projects go through these stages in a spiral: as time goes on, large sections will get re-written to update architecture, move to a new technology stack, fix the underlying issues of the old code base. And then more bugs will be written, tweaks and updates will be needed and will tarnish the shiny new design, and environment will continue to change.

No matter how many times we’ve been there, for most large successful projects, a large portion of the lifecycle is when the code base is full of slow-moving, bug-prone legacy code that is hard to work on. Eventually, an investment will be made to rewrite or refactor the worst pieces, yet a short while later the development organization finds itself facing the same problems again.  

At the same time, developers are being continuously schooled on writing better code, doing TDD, working with legacy code, but all this knowledge turns out to be very hard to put into daily practice.

Legacy code keeps happening, despite the best intentions.

Are we ready for code review?

We have been talking about the better ways to do code reviews, and there are plenty. There are great tools to make code sign-off by someone other than the code author a required part of the process. It is important to keep code reviews short and casual. Very little code and only a few people should be involved in each event. Reviews should be done often, as much as a couple of times a day, and everybody on the team should participate regularly. There are ways to limit the conversation to the most important topics – logical flow, proper use of language features, good naming, while leaving formatting, test coverage, and many other issues to the automated code checkers.

The most interesting conversation, as it typically happens, took place after the formal presentation. The biggest problem of code reviews, and all other technical reviews, is that we, the developers, tend to take the critique as a personal assault.  People brought up multiple instances when they tried to communicate code feedback to a colleague, and instead their comments were heard as snarky and demeaning. We discussed learning to be nicer, working on improving Emotional Intelligence and social skills, making sure to provide comments on good things as well as bad.

However, this is just one side of the problem. There are at least two parties involved in delivering feedback: a person who endeavors to give the message, and the person who is to get the information. The person who receives feedback must be open and involved into how feedback is provided. In order to learn from and benefit from feedback, we also need better social skills and EQ, as well as trust in the system and the person who is delivering the feedback. 

OO is getting old and tired. But is it dead yet?

Here are a few thoughts inspired by an excellent presentation with a provocative title “Object-Orientation is Dead, too” by Dave Thomas aka @pragdave.

OO has been taken to mean classes, inheritance and polymorphism, as implemented in a variety of programming languages. As we typically build code, the proper class design is to commingle state and behavior. Inheritance provides a simple way to vary some but not all aspects of behavior on connected objects. And polymorphism provides a way to package different behaviors into look-alike syntax.

Together, these features make for an incredibly flexible system, with many different ways to design abstractions, express dependencies, and store state in many objects across the codebase. Other words for flexible are ‘complex’ and ‘complicated’. The notion that having all this flexibility is good and powerful and smart encourages mixing a variety of abstractions and implementations, and long and wide dependency trees. As systems and corresponding codebases grow over time, accumulating ideas and approaches from many people, they grow more fragile, more tightly-wound. Code also becomes more dependent on the past history, rather than the latest, and therefore best, understanding.   

We know all that because OO had had a great run in the last 25+ years. Java, in particular, has been one of the most popular programming languages since it came out in mid-90s, both by the amount of code written and number of people writing in it. .Net platform is heavily OO and is very popular as well. C++, while not exactly a strictly OO programming tool, is frequently used for OO-style development. Lots of complicated projects have been attempted, and plenty completed, using the OO paradigm. Many people joined the ranks of OO developers, people with varying amounts of education, imagination and cleverness.

In the last quarter century OO paradigm has been applied to many complicated development projects, at times by less-than-stellar developers, and often by diverse groups working independently and inconsistently. OOP came out somewhat scathed, bruised by the many broken abstractions, blemished by millions of lines of legacy code, but by and large OO has delivered on its promise to provide a way to make large and complex systems possible.

So, is OO dead yet? Or, rather, are we ready to move on to something newer and better?

It’s been many decades since OO first showed up, handily won over then-popular procedural programming style, and far eclipsed the popularity of the functional approach. We are now older, wiser, and have more experience. Is OO still the silver bullet it has once been? And what are our options?

Functional languages are powerful, deeply loved by their communities, and offer a paradigm that can rival OO tools in solving complex problems. Yet, the functional approach has not enjoyed nearly as wide an adoption among broader development community. It also did not have such a testing 25+ years, as software projects grew larger, more complex, and continuously involved larger numbers of less skilled people than ever before.

The functional paradigm is worth exploring again, with the experience we did not have back when OOP originally came along and took over the software industry. But the winning paradigm for the next quarter century is far from certain.  

Solving for Passion

For the last few years, I have had the pleasure, and the privilege, to participate in graduating students’ project reviews at a major university.  Students in their final semesters of earning a Computer Science or Computer Engineering degree are asked to come up with and implement a project of their own choosing, within the guidelines set forth by the professor for the course. Students typically identify a user or a client for their work – a university department or research lab, consumers, charitable organizations.

A few dozen projects I have reviewed so far have mostly been spectacular. Students put their heart and soul, and considerable skills and passion for the technology, into this assignment.  However, there are interesting differences in the presented work.

Some cohorts of students appear to have lofty, large-scale goals. They are interested in solving big important problems related to hunger, or low productivity, or deadly deceases for large populations using tiniest of tools. The teams pull research from international conferences and large NGOs, make highly speculative assumptions about complicated unknowns, and pull together technologies that are both rare and not really designed for these purposes.

For these projects, results are typically modest, presentations are ambitious on vision and poor on structure, and risk of running into crippling unknowns is high.

Other groups appear to be much better prepared. The projects capitalize on the cutting-edge technologies, tests are neatly recorded on video, and presentation slides showcase the scientific background and beautiful technical documentation.  However, these projects aim a bit lower: gadgets that offer a small improvement on the existing products.  Polish replaces passion.

When the focus is on delivering a good-looking presentation, creativity gets lost, or at least greatly diminished. This batch of working, well-documented projects lacks excitement, grand vision, reach into the unknown.

College is the time to dream, to stretch for the moon, to grow one’s ambitions. Most people go on to have a long and very practical career, where delivering a small improvement is valued higher than big ambitious undertakings. We will all be better off if we had more passion and creativity to go around. 

Beyond the 'happy path'

In the context of software or information modeling, a happy path is a default scenario featuring no exceptional or error conditions, and comprises the sequence of activities executed if everything goes as expected

- Wikipedia

Software is possible because we have clear, well-defined expectations in a lot of common situations. Given a problem and a particular state of the world, software can reason through known information, and perform according to the expectations.

People who invent software are enterprising, excitable, focused on delivering a working solution for a problem.  Which mostly translates into building software that is great about performing the ‘can do’ portion of the expectations. It is the so-called happy path, where the software works as desired and designed, in the perfect world for the well-behaved user who happens to think in perfect lock-step with the inventor of the software.

However, the world is hardly ever perfect. People and users think differently. Not all users are both well-behaved, and can or will follow the detailed vision of the software designer.

Consider a bank application that allows read access to one’s account balance. It is important that it does NOT

• allow read access to other people’s account balance.
• allow write access to any account balance.
• allow any user behavior to affect other user’s experience and access.

These kinds of expectations are less likely to be included in the specification, because we like to think about software in terms of what it can do, and not what it doesn’t do. These expectations are less likely to be designed into the software architecture, because we like to think of achieving our goals must more, than we like to be concerned about preventing bad things from happening. Finally, these expectations are less likely to be tested, because it is both harder and does not demonstrate a job-well-done even if successful. 

Better handling the non-happy paths requires a more systematic approach to specification building and testing, and range verification.

Leading by good looks

“… you don't get trusted positions just because of your ability. You also have to attract the notice of superior[s…]. You have to be liked. You have to fit in with the system. You have to look like what the officers above you think that [leaders] should look like. You have to think in ways that they are comfortable with.

The result was that you ended up with a command structure that was top-heavy with guys who looked good […] and talked right and did well enough not to embarrass themselves, while the really good ones quietly did all the serious work and bailed out their superiors and got blamed for errors they had advised against until they eventually got out.”

Ender's Shadow, by Orson Scott Card @1999

Likable people that fit within the system do a decent job advancing the system toward success. The kind of success that is easy to imagine, easy to prognosticate, and mostly means being ahead of the competition that does the same thing.

However, if one wants to shoot for the stars, to change the paradigm, to redefine success – expecting good-looking people who think in ways that are approved by the existing system to lead us there is doomed to fail. Occasionally, success is not so much ‘the same thing, just more and better’, but rather a long shot that could turn out to become something different, powerful and amazing. And then there is a need for a different kind of leadership, the leadership that does not work as hard to look presentable, to say the expected thing, to be liked by the superiors.

There have always been unlikeable leaders who changed the world in big and small ways. A few went against the system, and others simply disregarded the system and built their own way. The society at large considered most of them failures during a significant portion, if not the entirety, of their journeys. Others have reached the official definition of success, and occasionally even became liked and popular. Still, leading for change is a risky proposition with high potential for failure.

So when picking a leader, or aspiring to lead, put some thought into what kind of leader you want to follow, or to become. Good looks may not be required. 

Following Plato Dialogues: conversation about estimates

-            So, why do we need estimates?

-          Well, as a customer, I want to have estimates.

-          Great! In what ways are estimates useful to you?

-          Say, I order from Amazon, and if my purchase does not arrive on the estimated day, I get to request a refund.

-          Huh, so you are rooting for the sender to fail, and that’s why you need estimates?

-          I guess so.

-          Any other reason to want estimates or tracking?

-          Well, if as a customer I was promised a certain delivery, and it is getting really close, but the work is less than half done, I can make a determination that I am not getting that delivery.

-          So, the reason you want estimates is to feel reassured that work is being done? To make you more comfortable with the delivery that you do not trust?

What are good and useful reasons to want estimates?