Sunday, July 30, 2017

Scrum as a series of feedback cycles

  • Daily scrum: every 24 hours, within the Scrum team. Feedback on technical details.
  • Sprint review: every 1 to 3 weeks, within the product team. Feedback on requirements, features delivered, and implementation details.
  • Retro: every 1 to 3 weeks, or, better yet, on as-needed basis, within the Scrum team. Feedback on communication, processes, resources.
  • Backlog grooming: every 1-2 days or even more often, within the product team. Feedback on features delivered, technical details communicated, and user feedback.
  • Sprint planning: not a feedback cycle on anything… why is it a part of Scrum?

Sunday, July 2, 2017

A key to the kingdom

Agile talks about emergent architecture and building vertical slices of functionality. It is typically understood that all efforts must go toward creating valuable user-facing pieces, as opposed to delivering software layers that cannot be used until all other layers are in place.  Unfortunately, this approach is often taken as “all we need is the visible parts”, i.e. the parts that the user gets to see and click on, or that push the data to be displayed.

When I first encountered a large DB supporting a well-used, living application with no indexes or foreign keys, I thought it was a rare oversight.  Then I encountered a few more database setups just like it, with large and heavily used tables and relationships, but neither indexes nor keys defined.  In all cases, users were complaining about slow response times, and administrators remembered more than one instance of the application becoming non-responsive under heavy load as DB requests timed out en masse.

Other developers I talked to also mentioned that they worked with reasonably large DBs that were missing indexes and foreign keys. A number of people, working in different companies on unrelated projects, mentioned that their engineering managers or software architects absolutely refused all suggestions to create even primary keys on tables “since nobody sees them anyway”. Several developers talked about applications being re-implemented, in part, because of poor response times caused by frequent timeouts of DB requests.

Vertical slices of user-visible functionality must include “the plumbing” that allows the application to handle the expected load. “Emergent architecture” is not complete without at least basic considerations of technical quality. A project cannot succeed without understanding what it takes to deliver user value in a production situation – and that typically includes multiple users working simultaneously.  Building in proper DB infrastructure is a simple example of such considerations, the work that is not visible but nevertheless valuable to the user.

Friday, June 16, 2017

When the math is off

How Developers Rate Their Own Programming SkillsMax Woolf @minimaxir

Here is a common interview question that is typically gets asked in interviews:
"Please rate yourself in your best skill, on a scale of 1 to 10, where 5 is average."

The answers most often are between 7 and 9.5 inclusive, with an occasional 10.
Lets consider what that means.

Skills and ability to contribute have been shown to be distributed according to a Pareto Principle: 20% of the population possess 80% of the skills and deliver 80% of the results. Suppose, it is a Pareto distribution with the slope of 1, and a minimum skill value of 1. There is no max value for the true Pareto distribution, but the way developers are asked to use the scale, the max value is set at 10.

A few runs on University of Alabama in Huntsville Special Distribution Simulator show that for a 1000-point simulation the mean is in the 7-9 range. So far so good.

However, the randomly generated values have no set maximum, and can go well above 10. Even a few higher values in a large set skew the mean significantly.  To get back to the allowed range of values 1 through 10, replace all values in the 1000-point dataset, that are above maximum allowed value of 10, with 10.5. The average drops down to ~4.  The median is ~2, i.e. half the data points have values of less than 2.

That contradicts the initial question, which sets 5 as being the average.  A distribution of skills is likely to have an average level skewing lower than half the range. In this example of modeling skill distribution with a Pareto distribution 1,1 - way lower, at the mean value of 2.  More importantly, it raises the question on how well the interviewers understand the question they are asking the candidates.

"The best and the rest: Revising the norm of normality of individual performance."
by Ernesto'Boyle Jr., Herman Aguinis

All of the above is just a long way of saying that when a typical candidate for a position rates himself in response to the question, they are, so to speak, gaming the situation.

The more interesting observation is that not only the question is asked regularly, by many, if not most, developers acting in the interviewer capacity. The answer is taken seriously and can influence how the interview proceeds, as well as the outcome. People with higher self-rating (those claiming to be above 8) are more likely to be hired and promoted.