Rules of software optimization

“Rules of Optimization:

Rule 1: Don't do it.

Rule 2 (for experts only): Don't do it yet. ” 

- Michael A. Jackson

The presenter talked about processing TBs of data in less and less time, even as models get more complicated. Slides covered different communication options, kinds of multithreading, benchmarks measuring time and memory foot-print for massively parallel processing. Chip capabilities and layout, high-speed interconnects, highly-specialized libraries and compiler options to take the best advantage of the available hardware.

Dealing with complex models on large data sets available all at the same time is a serious and very interesting problem.  Modern computers are immensely powerful, with a typical laptop packing multiple cores, popular languages and compilers doing a great job of optimization, and many high-performance tools widely available for learning and using. Yet most of existing and in-progress software only uses a tiny fraction of available horsepower, and the vast majority of developers are not aware of the vast amount of work their systems are capable of handling, with the right setup.

This is a fine state of things.

Optimizing performance comes at a cost. This cost can be time – it takes time to assess performance, find bottlenecks, and do something about them. This cost can be money – it takes a better-educated developer to measure and reason about performance, and what it should be. Most often, this cost also comes in correctness – optimizing code is tricky, and bugs can and will be introduced while chasing speed. The hard-core performance optimization, to take the best advantage of the available hardware and software, can make code longer and more complex, less readable and maintainable.

However, there is an even bigger reason why getting the best performance is only relevant in a narrow set of applications. A typical business application today is interactive, works with live user input, and produces output for human consumption in real time. Once output is produced fast-enough, there is no value in producing it any faster.  If showing a document on the screen takes 200msec, doing it any faster has no value.  

It is best to leave the performance optimization for the applications and scenarios that will deliver better value because of the speed. For the rest of modern software development, it is better to concentrate on correctness, maintainability, and making code as simple as possible.