Monday, September 25, 2017

Deciding the architecture

Agile project starts by gathering an Agile team, deciding on a project and specific deliverables to be built first, and setting up the needed infrastructure. 

But before the team goes about creating the infrastructure, several questions about the system architecture need to be addressed.  What infrastructure will be needed, and what skills the team must have, greatly depends on the proposed system architecture and technology stack.

Who decides system architecture is the “elephant in the room” issue of Agile projects.  Many smaller design decisions will happily “emerge” as the team goes through the daily work of building the project iteration after iteration. Still, there is a number of large-scale, long-lasting choices about system architecture, that must be decided at the start of the project, when relatively little is known and the team has the least amount of experience on the project.

Who should be making these decisions? The typical answer appears to be The Architect, someone high up in the ivory tower, someone who makes no mistakes by producing little to none tangible, and thus imperfect, results. Sometimes that person is very good, and in other cases they just get by.  As it turns out, making the very best initial architectural decisions for a given project is not particularly important for the project's success, and making passable architectural decisions is fairly easy and fool-proof. 

List of common architectural patterns is very small, and almost all projects fit nicely into that tiny set.  Even if an inappropriate pattern is picked, the team will be fighting, and winning, over the inappropriate pattern to create a working implementation.  If the system becomes successful despite poorly designed architecture, and fighting the system setup becomes a serious obstacle to evolving the product, it will get re-written with a less inappropriate architecture shortly.

Sydney Opera House is one of the most beautiful and distinctive buildings in the world. Its architecture went through a dozen iterations of design, and the finished structure exhibits terrible acoustics - a big problem for an opera house and performance venue.  The building took 10 years longer than planned, and went 1,357% over budget. Of course, the project was, and still is, a resounding success.

Sunday, July 30, 2017

Scrum as a series of feedback cycles

  • Daily scrum: every 24 hours, within the Scrum team. Feedback on technical details.
  • Sprint review: every 1 to 3 weeks, within the product team. Feedback on requirements, features delivered, and implementation details.
  • Retro: every 1 to 3 weeks, or, better yet, on as-needed basis, within the Scrum team. Feedback on communication, processes, resources.
  • Backlog grooming: every 1-2 days or even more often, within the product team. Feedback on features delivered, technical details communicated, and user feedback.
  • Sprint planning: not a feedback cycle on anything… why is it a part of Scrum?

Sunday, July 2, 2017

A key to the kingdom

Agile talks about emergent architecture and building vertical slices of functionality. It is typically understood that all efforts must go toward creating valuable user-facing pieces, as opposed to delivering software layers that cannot be used until all other layers are in place.  Unfortunately, this approach is often taken as “all we need is the visible parts”, i.e. the parts that the user gets to see and click on, or that push the data to be displayed.

When I first encountered a large DB supporting a well-used, living application with no indexes or foreign keys, I thought it was a rare oversight.  Then I encountered a few more database setups just like it, with large and heavily used tables and relationships, but neither indexes nor keys defined.  In all cases, users were complaining about slow response times, and administrators remembered more than one instance of the application becoming non-responsive under heavy load as DB requests timed out en masse.

Other developers I talked to also mentioned that they worked with reasonably large DBs that were missing indexes and foreign keys. A number of people, working in different companies on unrelated projects, mentioned that their engineering managers or software architects absolutely refused all suggestions to create even primary keys on tables “since nobody sees them anyway”. Several developers talked about applications being re-implemented, in part, because of poor response times caused by frequent timeouts of DB requests.

Vertical slices of user-visible functionality must include “the plumbing” that allows the application to handle the expected load. “Emergent architecture” is not complete without at least basic considerations of technical quality. A project cannot succeed without understanding what it takes to deliver user value in a production situation – and that typically includes multiple users working simultaneously.  Building in proper DB infrastructure is a simple example of such considerations, the work that is not visible but nevertheless valuable to the user.