Why do we suck at estimating stuff?

Background

You will find this funny now. I announced this article without having any clear idea of the answer behind the WHY question.

I have perfect examples of HOW we suck and WHAT to do to fix that. But WHY? Well, that’s what I planned on finding out as a direct consequence of having to write this article.

“Why can’t my developer provide a good and precise estimate”?

Is this you? Have you ever asked this question? Or wondered about it? What’s the freakin’ trouble of having to provide a reliable estimate and stick to it? Well, lucky for you, I have an answer; in a form of a question, though.

Let me ask you something and I really want you to pause and think about the answer before you proceed with reading. Ideally note the answer down on a piece of paper or whatever. You ready? Here we go:

How long does it take you to tie your shoes? *

Please don’t read any further before you give it a thought and note it down.

You ready? Perfect! Let’s proceed with next question now and again I’d like you to THINK about it first and ideally note it down somewhere. Here it goes:

How long would it take you to explain to somebody, who has never tied their shoes in their life, how to do so? **

Give it a though. I’ll wait.

You ready? Perfect! Let me share the stats now.

For the first question, people usually answer in a range of 4 – 10s. Give or take. The average value being around 5 seconds. Hence, pretty much everybody seems to agree on the same range.

Now, when it comes to second question, you’d be surprised that I’ve heard answers varying from 10 seconds to 8 years. I kid you not. And really, if you pause and think about it, that’s a hell of a difference.

The thing is – estimating software projects is pretty much similar to explaining shoe tying to someone who never did it in their life. Computers are really dumb as a rock. They can’t do ANYTHING without you telling them every single step that needs to be made. Their only advantage is that they are amazingly fast, so they compensate. But you still have to provide every single instruction to them!

*, ** These are not my examples, but I rather “borrowed” them from Robert C. Martin (Uncle Bob). He used these shoe tying examples in one of his presentations.

Why do we suck at it?

I took some time to think about and research this topic. And what I discovered is pretty much always the same — over-confidence, under-confidence, lack of knowledge, overlooking critical parts, ignoring that other things might come up in the meantime, yadda yadda yadda. That’s all cool and dandy but I’m sure you were already aware of this. Hence, I’ll give you a different perspective here. I’ll tell you what I think about this from evolutionary point of view.

I seriously believe that the whole reasoning comes down to the fact that our brains were just not made for doing it. Because, what’s the purpose of it, really? Just think in terms of evolution and what we needed to survive. Through most of the human history you didn’t really care about HOW much food, in kilograms (or pounds) you needed to survive the winter, right? But you pretty damn sure knew it in relative terms. Like, and I’m really pulling this out of my bottom now, but – you’d know that your family can survive for a week by dining on single boar. You didn’t really care HOW much it weights, but you were interested in a relative measure (1 big boar).

Hence, I argue that it simply boils down to our brains not being equipped with proper tooling for this. As a consequence of it, we had to develop measurement tools so that we can precisely measure stuff. But do you know what’s the most confident way of providing an accurate estimate for how long would it take to develop something? Writing the actual code behind it! Yes. You are measuring the lines of code that you need to write (and rewrite) in order to do something. Anything else is pure guestimating!

What do we do then? How do we estimate?

This is where all the fluff about Agile Frameworks and other stuff comes into play. Admitting to yourself that you are not equipped with proper inner tooling to estimate, you have to rely on external help.

So if you are tasked with estimating something (i.e. someone is asking you “how long would it take to do X”), here’s what you do:

1. Tell them that you will get back to them. Never-ever make any comments about the complexity (e.g. “well that looks easy”). Always tell the person that you will get back to them. And this is really because you HAVE TO put some thought over it. You need to think about it. You need to look at edge cases. So it’s not a bullshit but an actual thing – you HAVE TO think it through first
2. Now that you, hopefully, planned some time to think this through, what you need to do next is to try and think of any tasks that are as similar as possible to this one. Think it through, really. If you find anything similar then that’s a perfect reference point. You can communicate back that X looks similar to Y, and you think it will probably take around the same time
3. If you can’t find any similar examples, or if this is something completely new, then it becomes a bit trickier:
   1. If this is a whole project to be estimated, then by abiding to a methodology (e.g. Scrum) you should already be equipped with a framework for providing estimates. As in – create a backlog, split into smaller tasks, etc.
   2. If it’s not a project, depending on the size of the X, you either try to split it a bit, or you try and give your WORST and BEST estimates. Something like – I think this would take anywhere from 2 hours to 3 days. Forget all the crap with “buffers” and stuff. Just communicate worst and best scenarios and that should be enough.

And that should pretty much cover every possible case. I wrote a pretty extensive article that talks about Story Points and how to use them for estimates, so you might want to read that one as well.

But if you were to remember ONE thing out of this blog post, it’s that your FIRST answer has to be — “Let me get back to you with an answer”. Never ever ever communicate ANYTHING on the spot!

Useful Resources

• The Mythical Man-Month by Frederick Brooks – I’ve had this book for a while in my reading list. Haven’t read it yet but I definitely heard a lot of positive feedback on it
• When Will It Be Done? by Daniel S Vacanti – haven’t read this one either but I noticed that it was rated 5/5 from a person that I highly respect
• Story Points that mistook themselves for Hours by myself – it’s cheesy, I get that, but I do think it goes hand in hand with this one

Summary

• If you want to understand the pain of trying to estimate something, ask yourself the following question – “How long does it take me to tie my shoes?”. Once you have an answer, try answering this one – “How long would it take me to explain to somebody, who has never tied their shoes, how to do so?”. Observe the difference
• Dozens of reasons are floating around, covering the subject of “why do we suck at estimating”. Over-confidence, under-confidence, lack of knowledge, ignorance, fear, … Just check this google search and see for yourself
• I take a different approach and reasoning. I believe that our brains LACK the necessary machinery for doing this; primarily because it isn’t an evolutionary beneficial feature. Hence, we need TOOLS to do it!
• How to do it then? Ideally – use Story Points. If not possible, try and see if there was anything remotely similar to what you need to estimate. Finally, you just have to rely on your “intuition and expertise” (or, to use a scientifically correct term – pulling it out of your ass) in which case, at least consider Wideband delphi approach