In 2007 my colleagues at Construx Software and I updated the list of classic mistakes from my 1996 book Rapid Development. Throughout 2007 we conducted a survey to determine the frequency and severity of these classic mistakes. In other words, we wanted to get a more quantitative sense of just how "classic" these classic mistakes are.
More than 500 people responded to the survey. The majority of them were involved with web and business systems. A significant minority were involved in shrink wrap/commercial systems, and about 10% were involved in embedded, system critical, systems, SaaS, or other kinds of software. About half the respondents were in lead/architect roles, about one-quarter in individual technical contributor roles, and the rest were in management or dual management/technical roles. The results are available in a white paper, "Software Development's Classic Mistakes 2008." You will need a login our main web site to download the white paper. (The log in is free.)
Excerpts from the Classic Mistakes Survey
Based on the survey responses, we computed the approximate frequency of the mistakes surveyed. Here is an excerpt from the white paper that shows the approximate frequency of occurrence of the most common classic mistakes:
We also examined how severe the mistakes are when they occur. This excerpt from the white paper describes which mistakes produce Catastrophic or Serious consequences the most often:
Finally, we made an assessment of which classic mistakes are most damaging overall. We multiplied the approximate average frequency of each mistake times its average severity to arrive at a Mistake Exposure Index (MEI). The MEI ranges from 0 to 10, with 10 being the worst. Here is an excerpt from the white paper that shows the classic mistakes with the worst MEIs:
Here is an excerpt that summarizes the average frequency and average severity of the mistakes with the highest MEIs:
Conclusions from the Classic Mistakes Survey
The raw survey results are interesting and so are some of the general trends.
One conclusion is that two of the mistakes added in 2008 (i.e., that weren't in my 1996 book Rapid Development) made the top 10:
This suggests that continued refinement of the classic mistakes list is worthwhile.
A second conclusion is that a few of the mistakes don't occur frequently enough or aren't severe enough when they do occur to really be considered "classic" mistakes:
A third conclusion is that many of the mistakes in the survey do indeed deserve to be called "classic" mistakes. I find it interesting that 8 of the top 10 mistakes in this year's report were listed in a book I published in 1996. If these mistakes were classic in 1996, they're even more classic 12 years later!
Final Thoughts
We'll be updating the classic mistakes survey in 2009, and we'd appreciate your input into the survey. You can take the survey in about 30 minutes. If you take the survey, we'll send you the results before they're made available to the general public.
Why do people keep making these mistakes? I'm interested to hear your thoughts.
Resources