Software Best Practices

Okay, here's a really sticky situation for you.

For the last 3 years, I've been the sole developer on a software project that has gone notoriously awry. We're guilty of almost every one of the classic software mistakes. (For the horror fans, you can read about it here: Creating My Own Personal Hell.)

While we've made many mistakes in the past (most of my own making), I'm working very hard to correct the ones that I'm responsible for. A couple of those involve managing customer expectations regarding defect resolution times and testing times. The problem is, I have no real idea how to go about estimating that given that testing (as a science in and of itself) isn't really my specialty. (And we all know that developers make terrible testers.)

As described in the article, I have to write the requirements and the test plan, develop the software, test the software, log the defects, fix the defects, and regression test the system in a very short period of time. The customer, in this case, is very unreasonable about allocating sufficient time to testing. I'm trying to make a concerted effort to give reasonable estimates about defect resolution.

In retrospect, I think that one of the mistakes I've made in the past is in simply telling them that it will take X days or Y weeks to resolve the defects reported, without distinguishing between man weeks and calendar weeks. Consequently, the customer draws the inevitable calendar weeks conclusions. Another mistake I've made is not allocating time for the defects that turn out to be inordinately difficult to pin down. 

In this latest release, I spent a few days going through the system and tracking the time it took me to resolve a sampling of the defects. I determined that, on average, the resolution time was about 30 minutes. (Some took as little as 5 minutes, some were up to an hour.) So now I've got an estimate for the number of defects that we need to resolve, and I want to give them a best case scenario (number of defects divided by 2) and a worst case scenario (best case times 1.5). This gives me a number of man hours.

Is this a reasonable way to do this, or should I be trying something else? How do you guys do it? I am totally out of my element here.

Any help is greatly appreciated! 

Hi essentialgeek,

I moved your thread here so it can get some more attention!

I think you are on the right track though there might be a couple of other things you may want to do. First of all, to cover what I think you are doing right

• Collecting data about your defect detection and removal effort
• Counting (estimating) how many defects are outstanding
• Calculating a best case and a worst case

What might you add on top of that? I would suggest the follow

• Calculate a "most likely case" that is, somewhere 2/3 between the best and the worst leaning toward the worst (compensate for typical developer optimism)
• You can do a PERT calculation ((best+4*mostlikely+worst)/6) to get an expected amount of effort
• Decide, on average, how many hours a day you can reasonably dedicate to defect correction (as opposed to new functionality, meetings, etc.)
• Take into account any holidays, etc.
• Present it to the customer as X effort hours to be completed no sooner than Y date with a possibility of going later

Or, you can take an more "agile" approach in creating a backlog of work. Work in two week iterations and allow the customer to prioritize the work for each iteration while you get to say how long each sub-item of work will take (you can use the above or some other estimation technique). That way they self-coach about which defect will get fixed by what date.

Another approach would be to create a table that lists four or five categories of defects (small, med, large, oh-my-gosh). Then have different amount of time to repair them. Take any defect that are outstanding and classify them into one of the categories. Add up the resulting time. Be sure to include the uncertainty. You can do this several ways but the easiest is to give ranges for each of the categories.

Thanks very much for the feedback, Earl.

Your additional suggestions are all definitely doable, and sage advice to say the least. I'll see what I can do to rewrite this into a simple "script" (in Microsoft Word, of course) and use it as a guideline to go by in the future. I'll let you all know how it goes.

Thanks again! 

Keep us posted on how it is turning out! Perhaps some other have suggestions as well.

essentialgeek:

 (And we all know that developers make terrible testers.)

Not sure if this helps you in your current position, and maybe a bit off thread: 

What I have found developers to be very good at is Test Driven Development (specifically writing unit tests before coding) and performing code reviews, especially when given good instructions on how to do this most valuable activity. These two "testing" activities contribute significantly to the quality of the software. We also include these as part of our "Coding" activity in the WBS, not testing, so the customer doesn't see these as testing activity. To us, they aren't negotiable tasks (some exceptions of course).

Preventing defect injection is a bullet made of material that is shiny and metallic, but, alas, not silver.

Regards, David Crosswell