Software: Just Plumbing or Mad Science?

There seems to be a fundamental debate raging:  Is software more like mad science or plumbing?

This debate came to my attention via Mike Taylor’s article:  What Ever Happened to Programming? Mike’s argument is that programming is nothing more than plumbing today and it’s no longer fun.  He believes that there’s more fun to be had as a “mad scientist” developer, building everything from scratch with materials at hand.  So let’s look at the two major camps of this argument:

Software Developer as Mad Scientist

Reading the classics of software literature like Knuth, Brooks, et al, you get some notion of the programmer as hiding out in an arcane computer lab, late at night, pounding away at the keyboard as if to build some Frankenstein program.  Unlike Shelley’s creation, the outcome is far more benign and often even useful.  But the creation is always that:  pure construct from the mind of the programmer.  No assistance from the outside world aside from a few borrowed organs to create the Magnum Opus.

This image has historical truth in it–Knuth himself created TeX in a similar fashion.  Supposedly Woz and Jobs did the same with early models of Apple computers.  Every programming language we have at our disposal today clearly had some singular human force behind it:  Ruby, Haskell, Java, C, C++.  The list goes on ad infinitum.

These creations required intense and detailed knowledge of the hardware and operating system to create their monsters.  Whatever they required in their tasks, they often built from scratch by themselves.  They are the pioneers of our fields, the first wave of migrants on the digital frontier.

Without the Mad Scientists, we would be language-less, tool-less, and probably stuck with punch cards on ENIAC.

Software Developer as Plumber

I hear our job derogatorily compared to that of a plumber:  “We just put stuff together instead of build it”.  I don’t think that gives plumbing it’s due, nor does it really consider the rich history of the field.

Plumbing back in the late 1800s and early 20th century was a dicey business.  The entire practice was inconsistent, lacked any standard methods, and training was haphazard (for much more background, check out this article).

Appropriately, the National Association of PHCC (formerly the National Association of Master Plumbers), first met in committee in 1883 at the old Astor House, the hotel that provided the impetus to modern plumbing back in 1834. Many new plumbing inventions had appeared and too many plumbers were ill-prepared. Close on their heels would be the Mechanical Contractors Association of America, the American Society of Heating, Refrigerating and Air Conditioning Engineers and the American Society of Sanitary Engineering.

Wholesalers banded together, too, starting programs to prod manufacturers into standardizing such things as sink and basin outlets, faucet drilling, trap gauges, etc. The Central Supply Association, for example, was formed in 1894 and soon made contacts with the old Eastern Supply Association, the Plumbers Association of New England and the National Association of Master Plumbers. But it would take another 30 years to accomplish the standardization which everybody takes for granted today.

That means roughly the first 50 years of plumbing (1883-1925) was effectively like the Wild West:  Every man for himself and standards be damned.  The public often suffered as a result of this:

An outbreak of amoebic dysentery in Chicago during the 1933 World’s Fair was traced to faulty plumbing in two hotels. Tragic results were 98 deaths and 1,409 official cases. One year later, Major Joel Connolly, Chief Inspector of the Chicago Bureau of Sanitary Engineering, spoke these prophetic words:

“One of the lessons to be drawn from the amoebic dysentery outbreak … is that plumbing demands the very best, painstaking effort that thoroughly qualified, certified plumbers can give in every building, and especially where the systems are complicated and extensive, and where large numbers of people may be affected by contamination of water.” (emphasis mine)

Clearly standardization of materials, methods, and training gave plumbing a major shot in the arm for consistency, safety, reproducibility and public trust.  The plumber’s model started off as cowboy hacking of pipes in a haphazard way to a systematic method of standards, interoperability, training and licensing.  Along the way, there were glitches, problems, and issues.  Big surprise.  Sound familiar?

Our software legacy has taken us from raw register manipulation in assembly, through multi-generation languages (2GL, 3GL and god help us, 4GL), to huge amounts of frameworks, libraries and tools that give us unprecedented levels of productivity today that would be unheard of 10, 20 or even 30 years ago.  But the cost is that there is less of the low-level work to do, and more of the heavy-lifting at the business level.

We’ve complained, bitched, and moaned about spending too much time on things like low-level problems:  database connectivity, GUI frameworks, XML parsing.  And guess what?  People responded to those complaints by building libraries, tools and frameworks to make them happen. To give us what we always wanted:  the ability to focus on the business problems and not the lower-level constructs.  In essence, we’ve borrowed the plumber’s model.

You assemble the pipes, solder them together, solve local problems about how to route the sewer line around the funky wall shape, but you don’t get to set the pipe sizes, mold the elbows, or determine the ideal composition of solder for ease of melting.

You get to put things together for utility.  A large part of modern software development is nothing more than a utilitarian venture of “some assembly required”.

But even the first waves of migration to the American West by the military and trappers of the day had comparatively little impact to the settlers of the late 19th century.  And similarly, the mad scientists of software have a significant, but much smaller, impact in comparison to the plumbers of software.

So which are we?  Mad Scientists or Plumbers?

Both.  Neither.  It depends:  On who you work for.  On what you specialize in.  On where your interests lie.  There is a need for both:  mad scientists are the creators of new tools, frameworks, languages and OSes, plumbers are the integrators, the users, and the orchestrators.  Software requires both to survive.

This isn’t a question of what software is, but rather who you want to be.  But there are some facts that are hard to argue:

• There are more plumbers jobs than there are mad scientist jobs.
• The need for mad scientists seems to diminish over time, not because mad scientists are less important, but because more plumbers are needed once the mad scientists are done with their work.
• It’s very hard to be a good mad scientist OR a good plumber, but they are vastly different skill sets.

Be an mad scientist or a plumber, but don’t complain when you’re a plumber but you really wanted to stay a mad scientist.  The choice is, and always was, yours to make.