June 2007 - Posts

• Rumors of Software Engineering's Death are Greatly Exaggerated (aka Software Engineering Ignorance, Part II)

  A reader of my previous blog post on Software Engineering Ignorance pointed me to Eric Wise's blog post Rejecting Software Engineering. Eric seems like a bright guy, and he's a persuasive writer, but his post is another example of what I was referring to in my earlier post -- that is, people who are uninformed about software engineering spreading misinformation about it. One of Eric's arguments that is representative of other published arguments is that "software isn't like real engineering." But the "facts" he presents about real engineering are way off base. For example, he asserts early in his post that real engineering has "near perfect information on durability, composition, balance, etc.," but that claim is idealized and not correct. When John Roebling designed the Brooklyn Bridge, for example, the properties of steel cables weren't well understood, and so he used a safety factor of FOUR in designing the cable supports for the bridge. Obviously that is not even close to "near perfect information." Indeed, one of the hallmarks of engineering as opposed to science is that engineers will work with materials whose properties are not entirely understood, and they'll factor in safety margins until the science comes along later and allows more precision in the engineer's use of those materials. Eric's post goes on to use poor estimation in software as a point against treating software as engineering: "Look at the estimation problems we have in software." Again, this assumes an idealized and incorrect view of other engineering disciplines. Can you say "Big Dig?" The largest "real engineering" project in recent memory, the Big Dig was originally estimated to cost $2.6 billion. The final cost was about $15 billion. [Thanks to an alert reader for pointing out an error in my numbers, now corrected.] In Seattle (where I live), the construction cost of the Seattle Mariner's baseball stadium ended up being nearly double the original estimates. There have been many, many cases like this, which I discuss in my book, Software Estimation: Demystifying the Black Art. Estimation error in software is not any better or worse than it is in other branches of engineering -- the central issue is that estimating large, one-of-a-kind artifacts is always going to be subject to a high degree of error. Estimating the 40th similar house you build in a housing development is easy, but so is estimating the 40th similar customized version of a software product you deploy. Eric's post cites the fact that there are 10:1 differences in programmer productivity as an argument against treating software as engineering. Oddly, Eric cites drywall installers in support of this point, I guess to say that drywall installers are associated with traditional engineering. Here again, Eric needs to check his facts. Has he ever worked with drywall installers? I can tell you from personal experience that there is definitely a 10x difference between drywall installers, especially in terms of quality. Some guys install drywall in a way that makes it nearly impossible to texture well. Other guys get it right the first time, and texturing it is really easy.  The fact is that 10:1 differences in productivity and quality aren't unique to software, and so that fact doesn't differentiate software from engineering or from anything else. There was an interesting study conducted in the 1970s that found that the 80/20 rule applies in virtually every discipline: 20% of the programmers write 80% of the code, 20% of the police detectives make 80% of the arrests, 20% of the NFL quarterbacks make 80% of the touchdown passes, 20% of writers write 80% of the best selling novels, etc. Eric's observation that there are significant differences in productivity is valid, but it doesn't have any bearing on whether software is engineering. Eric invokes the work of David Parnas as an argument against treating software as engineering, and says Parnas's work in information hiding undermines the openness that is needed for real engineering. I honestly could not follow the logic of his argument on this point. Moreover, Parnas has been one of the earliest and most prominent proponents of treating software as engineering. Indeed, Parnas founded Canada's first undergraduate program in software engineering at McMaster University. Software engineering already has been defined as engineering, we have an international reference standard for that definition, the field's two largest professional bodies have jointly adopted a professional code of conduct for software engineers, we have accreditation standards for university programs in software engineering, we have university numerous programs that have already been accredited, and several countries are licensing professional engineers in software. Eric's conclusion that "I don't think software development will ever be able to be defined as engineering in the traditional sense" is a good example of the kind of uninformed opinion that I wrote about in my previous post on this topic. But I don't mean to pick on Eric -- no one can be expected to be well informed about everything. Eric's post simply highlights the fact that rumors of software engineering's death have been greatly exaggerated, and we need to do a better job of spreading the word that, in reality, software engineering is alive and well. Posted Jun 28 2007, 10:37 AM by Steve McConnell with 5 comment(s) Filed under: software engineering

• Software Engineering Ignorance

  The February 2007 issue of IEEE Computer contained a column titled "Software Development: What Is the Problem?" (pp. 112, 110-111). The column author asserts, "Writing and maintaining software are not engineering activities. So it's not clear why we call software development software engineering."  The author then brushes aside any further discussion of software development as engineering and proceeds to base an extended argument on the premise that software development is not engineering. I agree with the author that the specific act of giving instructions to the computer doesn't much resemble engineering. However, the fact that one software development activity out of many doesn't resemble engineering does not imply that software development as a whole doesn't resemble engineering. Numerous software development activities have clear counterparts in other engineering disciplines, including: Problem definition Creation of models to verify the engineer's understanding of the problem Feasibility studies to verify viability of design candidates Design as a central activity Creation of detailed plans for building the product Inspections throughout the product-creation effort Verification that the as-built product matches the product plans Ongoing interplay between the abstract knowledge used by engineers and the practical knowledge gained during construction etc.   This list could be much longer, but these items are sufficient to illustrate the point that, even though giving instructions to the computer doesn't have a clear counterpart in other engineering disciplines, many software development activities do have clear counterparts.  Taking a step back from the specific argument, I find it distressing that writers in 2007 are still propagating the myth that software development cannot be treated as engineering. We can certainly debate the value of treating software development as engineering, or software engineering's appropriate areas of applicability; but any debate about whether software development can be treated as engineering ignores the fact that it is being treated as engineering, and deeply so: The Computer Society adopted a Code of Ethics for Software Engineers almost 10 years ago. The IEEE Computer Society approved the Software Engineering Body of Knowledge 2.0 in 2004, which was adopted as an ISO/IEC Technical Reference 19759:2005. Curriculum guidelines and accreditation standards have been established for undergraduate software engineering programs. In the United States the official engineering accreditation board, ABET, has accredited 13 undergraduate software engineering programs since 2003, and in Canada 9 such programs have been accredited (by CEAB). Numerous provinces in Canada license professional software engineers, and professional engineers are chartered in software in England.  It's appropriate and useful to debate in what circumstances should software development be treated as engineering, or what kinds of software development work better when not treated as engineering, or what portion of software development should be treated as engineering, or how engineers in software should be trained, or what proportion of software developers really need to be software engineers -- but arguing whether it's possible to approach software as an engineering discipline is years out of date. What do you make of the fact that we can have a software engineering body of knowledge that has been adopted as an international standard (ISO/IEC TR 19759:2005), we have bachelor's degree programs in software engineering, we have accreditation standards for those programs, numerous programs have actually been accredited--yet people are still arguing whether software can be treat as engineering? Is the issue simple ignorance, or is it something deeper? Posted Jun 23 2007, 10:15 AM by Steve McConnell with 8 comment(s) Filed under: software engineering

• Classic Mistakes Updated

  In Rapid Development I wrote that, "Some ineffective development practices have been chosen so often, by so many people, with such predictable, bad results that they deserve to be called 'Classic Mistakes.'" That was in 1996. At that time I was self-employed and most of my experience had come from working with only a handful of companies. New Classic Mistakes After founding Construx, a decade of work with hundreds of companies has enabled us to identify several new classic mistakes. Here are the additional classic mistakes we've identified: Confusing estimates with targets Excessive multi-tasking Assuming global development has a negligible impact on total effort Unclear project vision Trusting the map more than the terrain Outsourcing to reduce cost Letting a team go dark (replaces the previous "lack of management controls") Next Step: Hard Data on Classic Mistakes  The next step in our work is to identify just how "classic" these classic mistakes really are. Are they really made very frequently, and is the impact really very bad when the mistakes are made? To answer those questions, we've launched a Classic Mistakes Survey, and I invite you to take it. The survey lists 42 classic mistakes and asks you to rank the frequency and severity of each mistake in your experience. When we have enough responses we'll post the results of the survey. People who complete the survey will receive a summary of the survey at least 30 days sooner than the general public. So please, take the survey! The 36 Original Classic Mistakes For the record, the table below lists the original classic mistakes from Rapid Development. And here is a link to the full text of the original Classic Mistakes chapter from Rapid Development). People-Related Mistakes   Process-Related Mistakes   Product-Related Mistakes   Technology-Related Mistakes 1. Undermined motivation 2. Weak personnel 3. Uncontrolled problem employees 4. Heroics 5. Adding people to a late project 6. Noisy, crowded offices 7. Friction between developers and customers 8. Unrealistic expectations 9. Lack of effective project sponsorship 10. Lack of stakeholder buy-in 11. Lack of user input 12. Politics placed over substance 13. Wishful thinking 14. Overly optimistic schedules 15. Insufficient risk management 16. Contractor failure 17. Insufficient planning 18. Abandonment of planning under pressure 19. Wasted time during the fuzzy front end 20. Shortchanged upstream activities 21. Inadequate design 22. Shortchanged quality assurance 23. Insufficient management controls 24. Premature or too frequent convergence 25. Omitting necessary tasks from estimates 26. Planning to catch up later 27. Code-like-hell programming 28. Requirements gold-plating 29. Feature creep 30. Developer gold-plating 31. Push me, pull me negotiation 32. Research-oriented development 33. Silver-bullet syndrome 34. Overestimated savings from new tools or methods 35. Switching tools in the middle of a project 36. Lack of automated source Take the survey! Posted Jun 15 2007, 10:07 PM by Steve McConnell with 7 comment(s) Filed under: classic mistakes

• Estimation of Outsourced Projects

  A question we sometimes hear from our clients is, "My company does outsource software development for other companies. Is there anything special about estimating in that context?" There actually are some distinctive aspects to estimating in the context of preparing a bid or price quote, and I don't discuss that in my book Software Estimation.   Estimation in a Time & Materials Context   Creating estimates to support time and materials bids (i.e., charging by the hour), is only barely a special case because the very structure of T&M implies some variability in the outcome, same as my recommendations for estimating in-house development work. The only real difference if you can even call it a difference is that you have to make doubly sure that you're setting expectations clearly: "This is an estimate. We can't know the outcome with 100% certainty. Actual results will depend on exact details of what you end up requiring and how different issues get prioritized throughout the project," and so on.   Estimation in a Fixed-Price Context   In contrast, estimation in a fixed-price context is very much a special case. If your estimate causes you to bid too high, you won't get the work. If it causes you to bid too low, you will lose money. Both of these are undesirable outcomes! In other circumstances I usually find myself recommending that people back away from really elaborate estimation approaches because there's so much inherent variability in software projects that the accuracy of your estimates is inherently limited, and you reach the point of diminishing estimation accuracy after you've put in even a little bit of effort. But a fixed price environment, at proposal time, is one of the few circumstances I've run encountered in which an elaborate estimation approach is warranted. And so my first recommendation is, If your business depends on creating fixed price bids, focus on estimation skills as a core competency and treat estimation work as a business-critical function. That means, read Software Estimation, take my company's estimation class, and read other people's estimation books.   My second recommendation is similar to my general recommendation that you separate the "estimate" from the "target." In a fixed price bid context, separate estimation from pricing. The estimate informs the price you'll charge, of course, but there isn't any necessary relationship between the two. You can price a bid at the "unlikely" end of the estimation range if it's really important to you to win the work, and you're willing to lose money on it. Or you can price it way above the estimation range if you think you have an approach that allows you to perform the work at low cost to you and that delivers a higher value to the client.   We've seen lots of companies wrap themselves around the axle when the sales staff insists on lowering the "estimate" to get the work, when really what needs to be lowered is the price. This creates confusion throughout the project. Giving everyone permission to keep estimates and prices separate increases accountability on the sales side (they have to own up to the fact that they're pricing something on the low end of the estimation range and get buy in to do that), and it improves planning on the dev side -- if there's a big gap between the price and the estimate, the project needs to be treated as a higher risk project than if there isn't a large gap. When estimation and pricing are merged into one concept and called "estimation" (even though it isn't really estimation), the project planners can lose the important risk information that arises from the relationship between the price and the estimates.   Try not to do the "commitment/pricing" estimate until later in the cone of uncertainty. Of course this is the holy grail, but most companies can't do this with any regularity because they feel that the competitive pressures require them to submit bids in the wider part of the cone.   Bid smaller amounts of work when you can, i.e., be more iterative. One of the great benefits of iterative development is the ability to generate project-level data on early iterations that can be used to estimate later iterations with really good accuracy. The companies we've worked with have settled in on 3 iterations as the number needed to calibrate a project team's productivity. Interestingly enough, it doesn't seem to matter whether the iterations are 1 week or 1 month or longer -- it still takes 3 iterations.   Consider creating two-phase bids when you can. You can call the first phase "preliminary work", "exploratory phase," "proof of concept phase," "design phase", "Phase 1", etc. The purpose of this phase is to attack all the sources of high variability feeding into the estimates and ultimately deliver a bid for the second part of the project after the cone has been narrowed considerably. We've seen many companies use this approach successfully, although I can't think of any companies we've seen that have been able to use it for the majority of the work they bid on. Again, competitive pressures seem to lead to their using this approach only selectively.   Two phase bids can be structured either more "waterfallish" or more "agile". The description above assumes a more linear development approach in which you're trying to get most or all of the requirements defined up front and then bid the whole project. In a more agile approach, you can treat "Phase 1" as an actual design-build-deliver cycle, but structure it into 3 iterations so that you can get good project-level calibration data that you can then use as the basis for bidding the remainder of the project.   Collect historical data on your estimates at proposal time vs. the eventual outcomes so that you can build your own cone of uncertainty. The better records you keep about what materials fed into your estimate, the more meaningful your cone will be. For example, you might have really specific requirements for one bid and pretty vague requirements for another. In one sense, if they're both "proposal time" estimates, you might treat them similarly. But if one was supported by significantly more detail in the requirements that implies you're at a different location in the cone, and you'd want to account for that.   Non-Estimation Recommendations   On this particular topic, several of the most powerful recommendations aren't specifically about estimation; they're about project control.   Go highly iterative as early as you can, regardless of whether the bid is structured into one or two phases. Even if you're working to a single-stage bid, there's value to getting project-level calibration data sooner rather than later. If you discover 10% of the way into the project that you've under bid it by a factor of 2, you can go back to the customer sooner and reset their expectations, you can give the customer options that you still have time to act on, you can implement functionality in strict priority order, you can identify the project as a high risk project and manage it accordingly, etc. But if you don't have the project level data that tells you your initial estimates were way off, you'll just run the project as "business as usual", which is really the last thing you want to do.   Document assumptions at the contract level, spell them out in as much detail as you can, and then *contain* them. If you build a house, your building contractor might let you specify the kitchen cabinets, but there will be a line item in the contract budget for cabinets. If you end up choosing cabinets that are more expensive, you pay the difference. You typically would have line items for all kinds of things: lighting, landscaping, carpet, flooring, countertops, etc. The areas that are more certain (e.g., roofing, siding, foundation, plumbing) are simply specified. In software projects we also typically have areas that we can specify in detail and other areas that we don't know enough about at contract time to specify in detail. So in software contracts you can include clauses like, "The exact work required in the XYZ module has been budgeted at 40 staff hours. If work on XYZ exceeds its budget, the contract price will be increased correspondingly." I'm not an attorney so I am not recommending this as specific contract language, but hopefully this gives you a general idea about the general kind of clause you would ask your attorney to include in a contract.    Manage your set of projects/bids as an investment portfolio, accepting that some will "win" and some will "lose." From a theoretical point of view, if you're estimating early in the cone there just isn't a good answer to improving the accuracy of your estimates on a project-by-project basis. The fact is, your estimates will be off to varying degrees, and when you happen to get one that's pretty accurate it will be a matter of luck, not skill, because of the inherent limits of the Cone. On the other hand, assuming there isn't any bias in the early-in-the-cone estimates (which can be a huge assumption), you can essentially punt on the question of project-by-project profitabilty and instead focus on portfolio level profitability. Solving the problem of estimating accurately in the wide part of the cone for an individual project isn't even theoretically solvable. But solving the problem of estimating a collection of projects in the wide part of the cone IS solveable. The key to solving that problem is rooting out any systemic bias in those estimates so that the error tendency is neutral. Then with that set of neutral estimates you simply increase each estimate by the amount you'd like your profit margin to be. If you want it to be 10%, you bid 10% higher than your neutral estimate. This will result in your actual project cost coming in higher than some of your estimates and lower than others, but on balance, assuming no systemic bias, you should make a 10% profit on your portfolio of projects.   Of course this requires that you have several projects in your portfolio, and that there aren't just one or two huge projects whose estimation errors could drown out whatever error was contributed by the smaller projects, and that you can afford to take a loss on some percentage of your projects. And those are big assumptions that might not be true in your specific case.   Bottom Line on Estimating in a Fixed-Price Bidding Context   The bottom line on this particular question is that it isn't possible to solve this particular problem purely using estimation practices. You have to change when you're estimating (later in the Cone), or what you're estimating (e.g., portfolios vs. individual projects), or how many times you estimate (e.g., two-phase bids). And project-control responses (as opposed to estimation responses) and even contract-level responses will probably turn out to be at least as useful as estimation responses.   Resources My estimation book, Software Estimation My company's software estimation class, which I teach a few times a year, which is both fun and highly educational My company's free Construx Estimate estimation software Estimation consulting services--we've helped lots of companies improve their estimation practices Comprehensive list of estimation resources on my company's web site, including a link to our Cone of Uncertainty poster, Cost of Estimation Error poster, and numerous other resources   Posted Jun 06 2007, 07:59 AM by Steve McConnell with 6 comment(s) Filed under: estimation