Magic-8 ball, and other approaches to problem-solving

There is a complicated way to do complicated things. Think long and hard. Spend hours staring into the blank space, screen or a sheet of paper or a list of specifications, or what-have-you. Then spend more hours furiously typing or writing or arguing out loud, while pacing nervously. Grab heavy, plain-jacket books with no pictures and very small font, throw them around. Doodle, draw up UML charts, and write down dozens of numbers… Do all of the above in freshly dry-cleaned business attire, or PJs, or worn-out jean shorts, or nothing at all.  Repeat until half hour before the deadline. Feel very excited about the results. 

This is pure magic. It is shrouded in mystery and emotion, takes magical amounts of time, and delivers unpredictable results with unknowable certainty. When it succeeds, it triumphs heroically, and when it fails, it does so with a big loud bang.  

There is also an easier way. Make up something, anything, however wrong it may be. Devise a test, and perform it. Go back to the drawing board, the text editor, the spreadsheet, etc. Notice the problems in the previous idea and implementation. Then try again, taking the new information into account. Update the test as well. Continue until happy with the results. Heavy books, typing furiously, and worn-out jeans may still come in handy, but this way those are means to an end, not a pure expression of experiencing a difficult and unfair world.  

When a complicated problem is solved by taking a small step at a time and adjusting, there emerges a combination of two possible solution paths. One path is as expected: iterating over small steps leads to a locally optimal solution. The achieved solution is local to wherever the initial, the least educated and therefore the most random, guess happened to be. In a lot of situations, this is good enough: achieved solution is proven to fit the problem by a series of tests, it is achieved in a reliable and predictable way, and no magic is required.

The other path is less expected, but nevertheless fairly common. Making small steps in the possible solution space allows to build an intuition about the underlying patterns and relationships. In addition to a set of concrete, tried-and-tested solution points, there emerges a deeper understanding of the problem. This leads to an intuition that allows to escape the local maximum, and helps reason about the entire solution space. Furthermore, a test-driven approach gives additional freedom to try out and gain confidence about further-fetched ideas in a systematic way.   

The iterative approach is simple, rational, and pragmatic. It succeeds routinely and quietly, and fails rarely and cheaply. Magic may still happen, but it will be more grounded and less dramatic.  More importantly, magic is not a requirement for success. 

How magical do you want your problem-solving to be?