Your Engineering Estimates Are Setting You Up To Fail

The 3 Questions That Kill Bad Estimates

The TPM, let's call him Terry, was furious. I was sitting in a conference room with about 20 other people, and Terry just tore into me, my team, and our abilities. We were late, again. We missed our own estimates, again.

He was right.

Over my career, I led software projects with strict, hardware driven deadlines. Or complete software projects with no external goalposts, but with dozens of teams depending on each other for work.

Here's what you need to know about estimating software projects:

1. 90% of your estimations will be wrong

2. Estimations are a form of self-fulfilling prophecies

To save you from learning the lessons that I learned the hard way, I put together this two-part guide. In today's post, I'll share:

1. The 3 Power Questions of Software Estimation

2. How (And When) to Estimate for Accuracy

In the second post you'll learn how to take these estimations and work effectively with your partners and stakeholders. A common mistake about estimation is that it's a one-off process. You do the estimation once and then you forget about it. Nope.

You'll also learn about the silent danger of buffers. But all that’s for next time.

Now let's get started!

(Stick around till the end, there's a bonus spreadsheet.)

The 3 Power Questions of Software Estimation

I once supported a large, cross organizational project with a deadline set in stone. It was one of the projects that no one wanted to do, wasn't shiny at all, and yet absolutely couldn't fail. The cost of rushing it and compromising on quality was measured in hundreds of millions of dollars. To make it even more challenging, there was a clear deadline that couldn't be moved. It wasn't a matter of escalating to the right executive, it just couldn't be moved.

On the other end of the spectrum, I've had countless projects that just didn't matter. They were either dead on arrival or died quickly after we started working on it. Sometimes it was a premature ask from a partner, while at other times it was just deprioritized as more important work came in through the pipeline of incoming requests.

This leads us to the first Power Question: "What problem are you trying to solve?"

The First Power Question

Before you spend even one minute estimating how much effort a project will take, you have to understand why the project ask made it your way. And double down on the estimation part, why do you even need to make an estimate? How will they use your answer?

Power Question #1: What problem are you trying to solve?

This question can lead projects asks straight to the trash bin. And they can turn a one-week focused estimation workshop into a two minute "guestimate."

I really can't stress this part enough. Most estimation effort is wasted effort.

It is very rare to need a very accurate estimation of a software project.

But for now, let's assume that you do need to estimate with some measure of accuracy. How do you go about doing it?

This brings us to our next two Power Questions. While you addressed the first Power Question to your manager, product counterpart, or partner, the next two questions are engineering focused. You should ask your engineers these questions. Or, if it's a quick estimation that you can do yourself, then you should work through these questions on your own.

The Second Power Question

The second Power Question is: "How long will this take, if anything that can reasonable go wrong, goes wrong?"

This question is not about pessimism. Rather, it's about addressing reality as it is.

Here's how the typical software engineer makes a software estimation:

1. Break the project down into tasks

2. Estimate how much each task takes

3. Add everything up

4. Add a buffer at the end (if they've been around the block)

And this is exactly the wrong way to estimate. Because steps 1-3 are the most optimistic scenario, and step 4 is a complete guess.

Power Question #2: "How long will this take, if anything that can reasonable go wrong, goes wrong?"

Your job is to nudge your team towards the likely ways that things go wrong. Not that aliens attack and blow up your data-centers, but rather the likely scenarios that happen every day in tech companies. The person going on an extended leave. The integration costs. The unexpected data discrepancies before switching to the new system.

What you want your team to do is:

1. Break the project down into tasks

2. Brainstorm how each task can go wrong

3. Brainstorm how the transitions between tasks can go wrong

4. Add everything up

This gives you the "reasonable worst case" estimate.

The Third Power Question

The typical software engineering estimation process is broken due to another subtle reason. It is not optimistic enough. Sometimes, things just go smoothly. Sometimes, work you expected to have to do becomes a no-op. Maybe you found another team that already developed a solution you can reuse. Or a requirement ends up not being important and is dropped.

And this leads us to our last Power Question: "How long will this take if everything that can reasonably go well, goes well?"

This is the optimistic, but reasonable scenario.

Power Question #3: "How long will this take, if everything that can reasonably go well, does well?"

Now we have three numbers: the optimistic case, the likely case, and the worst case. It's time to put them together.

How (And When) to Estimate for Accuracy

Leading a software team is hard. Estimation is just one of the many challenges that managers have to deal with. It can feel overwhelming because the amount of work is never ending. And even when you do everything right, you can still find yourself, as I did, being roasted in front of twenty people in a conference room.

This is why stakeholder expectations is so important, which will be the topic of the next post.

Okay, I lied.

Yes, there will be a second post. But I already gave you the most important advice on how to manage stakeholder expectations. It's in that first Power Question. What problem are they trying to solve. This will not only help you invest the right amount of effort into the estimation process itself, it will also help you communicate the estimation effectively.

I love Power Questions and systems thinking. There is only so much information that I can share in these posts. If you'd like to learn directly from me together with a group of diverse peers from across the world and the industry, then you should check out The Manager Operating System. It's a two-week cohort based course that a past student called:

Gilad's course for managers was honestly the BEST one I've been to (and I've been to a few...)

If you can't make this cohort, I get it. That's why I'm trying to share as much information for free here in the newsletter as I possible can. Like what to do when you really need to estimate for accuracy.

The Estimation Algorithm

Right now you have three numbers:

• The best-case scenario

• The most likely scenario

• The worst-case scenario

To translate this into a number that you can share with stakeholder, use the PERT distribution:

• Estimate = (Best Case + 4 * Likely Case + Worst Case) / 6

This formula gives you a weighted average of the three numbers. But you shouldn't stop there, you need to calculate a second number:

• 95% Confidence Estimate = Estimate +/- 2 * (Worst Case - Best Case) / 6

These two numbers together give you a more accurate message to share. It's not just the estimated completion date, it's also the confidence range that you have.

Now, there's one point here that I glossed over that is important, and is at the heart of getting to an accurate estimation.

You compute these numbers for each task, not just for the overall project. This then gives more accurate numbers for the project overall:

• Estimate = SUM_PER_TASK(Best Case + 4 * Likely Case + Worst Case)/6

• 95% Confidence Date = Estimate +/- 2 * Square_Root(SUM_PER_TASK((Worst Case - Best Case) / 6)^2)

If you're worried about the math here, just copy this Google Sheet template and you should be good to go.

What if It's Not Accurate Enough?

Sometimes, that +/- number can kill your estimation.

Sometimes, you really do need a very accurate number. There are teams that depend on you being done by a certain date. Or there is a large marketing event that can't move.

There are three ways to increase your estimation accuracy that actually work.

Your first option is to spend more time on the estimation. Break down the project into smaller and smaller tasks. When you do this, when you increase the resolution with which you evaluate the project, the space where things that can go wrong shrinks. Likewise for the pleasant surprises that may happen. And the natural outcome is that you will get a tighter bound around your estimation.

The second option is to change how you sequence the work. Your initial estimate remains, including the broad boundaries. However, you can front-load as much of the uncertainties to the beginning of the project. Ideally, you work on the tasks with the largest standard deviation. Often, this just won't be possible because some tasks depend on other tasks. In this case, you can add another task, early in the project, whose sole purpose is to reduce the uncertainty of a later task. You're adding work to the project in exchange for more clarity, earlier on.

The third option is to cut to the bone. Ask yourself the bonus Power Question: "If I had to do this in 1/10 of the time, what would I do?" What tasks could you just not do? How could you hack your way around the tasks that seem like they are unavoidable? Can you be more ruthless in prioritization?

What if It's Still Not Accurate Enough?

The truth is that even after you utilize the three tools above, your estimation will still not be accurate enough. Real accuracy only comes through getting down into the weeds and going through the details.

• Which tasks can only be done by specific individuals?

• What dependencies between tasks are unavoidable?

• What do you need from external stakeholders, and by when?

• What are the biggest risks at each stage of the project? How can you detect them early? What can you do to mitigate them?

Many people don't like to hear it, but it really is possible to come up with an accurate estimate for software projects. It just requires work and effort. In nearly every case, you can better allocate this time to more productive work. In the rare cases when you really do need to know ahead of time when you will be ready, there's just no avoiding investing real engineering work into the estimation.

Luckily, that is the very rare exception.

Your Toolbox

There's so much more to say about project estimation. For now, here's a recap of what you now have in your toolbox:

1. Three Power Questions to pierce through noise and spark creativity.

2. A fourth bonus Power Question.

3. The Three-Point estimation template that is the basis of effective stakeholder communication.

4. Three concrete ways to increase the accuracy of your estimation, including adding more work.

I built and refined these tools over years of leadership, both in Big Tech and in startups. What started as uncomfortable roasting from Terry ended as ... quiet. No drama. No fire drills. Just calm execution.

In the next post you'll learn about the silent danger of buffers and how to manage stakeholder expectations.

Thank you for reading this post! If you want to expand your management toolbox even further, check out the Manager Operating System.