Blog

The latest insights from 50 techies who love what they do.

Leadership Best Practices for Migrating Legacy Applications

Migrating enterprise software systems to modern languages, platforms and libraries is a deeply challenging process with many risks and matching potential rewards.

DOwnload Resource

Optimism, Creativity, and Delays in Software Delivery

“Making good cooking takes a while. If you are made to wait, it is to serve you better, and to please you.”

This quotation was, at one time, printed at the top of the menu of Restaurant Antoine in New Orleans. The identity of the author is lost, which to me gives the statement an apocryphal quality that makes it even more universal.

 Whomever it was that penned this statement:

  1. Has my utmost admiration and respect. (This is a creative person standing up for the integrity of their art.)
  2. Is a certified bad ass. (They’re essentially telling their customer: “I got this. Sit down and shut up.” The French makes it more palatable.)
  3. Would not have enjoyed a career in software development. (If you’re reading this blog no further explanation is required.)

The quote is the opening epigraph to Fred Brooks’ essay on software effort estimation, titled “The Mythical Man-Month.” Originally published in 1975, the essay’s central theme is that large programming projects always take longer than we think, and that piling additional people on the project will only delay it further.

The fact that Brook’s observations still ring true over 40 years later produces some grim laughter on my part. Every developer who has spent more than a hot minute in the industry knows that the specter of time looms over every team member, status report, story point and deliverable of every software project, ever. It reminds me of Tom Hanks’ character’s oft-repeated motto in the film Cast Away: “We live and die by time.” Being a part of a software development team often feels the same way.

A programmer considers how to respond after being asked “What’s your status?”

In his essay, Brooks offers five reasons why the delivery of software projects is so often delayed:

  1. Industry estimation techniques are poorly developed.
  2. Effort does not equate to progress, but is often mistaken for it.
  3. Programmers are optimists (especially under pressure).
  4. Schedules are poorly monitored.
  5. When slippage occurs, the immediate response is to grow the team.

There is a lot to unpack here, so today I want to focus on Developer Optimism and how it effects estimates.

Brooks presents a passage from The Mind of the Maker, by Dorothy Sayers that posits creative activity can be divided into three stages: the Idea, the Implementation, and the Interaction.

“A book, then, or a computer, or a program comes into existence first as an ideal construct, built outside time and space, but complete in the mind of the author. It is realized in time and space, by pen, ink, and paper, or by wire, silicon, and ferrite. The creation is complete when someone reads the book, uses the computer, or runs the program, thereby interacting with the mind of the maker.”Brooks’ point in sharing this passage is to underscore the fact that we so often forget: estimation is not actual. Ever. No matter how long we spend in the idea phase, inconsistencies and gaps in our ideas only become clear during Implementation. This is why authors work through several drafts of their stories, and why Martin Fowler & Friends wrote The Agile Manifesto.

Not even Kerouac got it right the first time.

Most people would admit that the principle Brooks lays out is a simple truth: you won’t know how long it will take to do something until you finish doing it. A question I’ve often heard leveled at software developers is, “But, you’re experts at this, right? Shouldn’t you know how long it will take?” (Often followed by some cross-industry comparison to building a house or a car.) The answer, “Well, yes and no”, should be as familiar to developers as the question.In other professions, the medium with which objects are created is often a physical thing, an object in and of itself, with known properties. Those properties present a set of understood problems: the cake fails to rise, the battery goes dead, the pencil lead snaps. The creative industry professional knows this and can plan a logical contingency, such baking an extra cake or having more than one pencil at hand.

The same is not true for computer programming, Brooks argues. As developers, our medium is completely flexible and doesn’t exist in a physical way. Development is a kind of modern magic; we spin application gold out of binary straw. What Brooks suggests is that because the medium of programming is tractable, developers erroneously expect few headaches during creation. The thought being that since the medium can ‘do anything’, so can we.

Software development is unique as an industry and doesn’t match up well to other engineering or creative professions. For one, the costs and the stakes are much higher. If a cake doesn’t rise, a birthday is ruined. If a software project fails, it could mean fired staff, millions in lost revenue, or any number of real-world catastrophes. Imagine the reaction if we told our clients to pay us twice so we could build two versions of the application, in case one misses the mark!

To top it all off, software development managers are people pleasers. When pressed about “getting it done faster” from a client (or their own boss), managers are very likely to end up agreeing to unrealistic expectations; even if the estimates presented by their team say otherwise.

Brooks calls this Gutless Estimation, and states “[For] the programmer, as for the chef, the urgency of the patron may govern the scheduled completion of the task, but it cannot govern the actual completion. An omelette, promised in two minutes, may appear to be progressing nicely. But when it has not set in two minutes, the customer has two choices—wait or eat it raw. Software customers have had the same choices.”

Even if I find Brooks’ label a bit harsh, the statement rings true, which is precisely why I respect the apocryphal author of the Restaurant Antoine menu so much: his or her display of backbone in the face of paying customers is one I wish our industry accommodated. All too often we are pressed to reduce our estimates in order to make timelines, to close a deal, or even just to end the debate with a manager in a meeting (and let us get back to actually doing the development!)

There are many reasons for delays, and Brooks’ essay does a fabulous job of breaking the reasons down in further detail–and in providing solutions. One contributing factor to this is optimistic estimation, which can create a perceived “delay” when the actual estimate may have differed.

In summary, Developer Optimism happens because:

  • The scope of creative work is not clear until we complete implementation.
  • The medium of programming lends itself to a “nothing is impossible” mentality.
  • Industry, client and managerial pressure discourages developers from “sticking to their guns”.

Brooks proposes two solutions:

  1. The industry should continue to develop and share productivity figures, bug-incidence rates, and estimating rules so everyone can benefit as we search for a standard.
  2. Developers of all ranks need to “stiffen their backbones” and defend their estimates.

What do you think of Brooks’ arguments and his proposed solutions?

Read Fred’s essay “The Mythical Man Month” online via Drexel University:

https://www.cs.drexel.edu/~yfcai/CS451/RequiredReadings/MythicalManMonth.pdf

Buy Fred’s book of essays on Amazon:https://www.amazon.com/Mythical-Man-Month-Software-Engineering-Anniversary/dp/0201835959