Estimation does not help us learn how well our software will meet its aims; putting software into use does.
Software Estimation is a response to a range of often unexamined questions, and it's rarely the best answer to any. Yet many hold onto it like a life preserver. Why it falls short and alternatives.
I contend that software estimation is being applied in software product development more than is needed. Software estimation is deployed too often as a default practice for which alternative practices, including just getting on with it, are better options for addressing the different needs software estimation is supposed to serve.
This is not my first post addressing software estimation, nor is it likely my last. There’s plenty of criticism of software estimation, including its significant inaccuracy, its penchant for being used in a way that claims greater precision than is possible, and the fool's gold quest that the only way to improve was to ‘estimate better’. Indeed, I may be accused of overdoing it, like this classic scene from the British TV classic comedy ‘Bottom’ in an episode named ‘Gas’:
Now, I will caveat my statement on the overuse of software estimation by acknowledging that the degree of overuse may differ in different contexts depending on factors such as commodification and the need to coordinate. Still, in general, across contexts, I believe it to be true that significantly more time is invested in estimation than needs to be. In an age where the tools and practices to establish useful feedback loops exist, there are better means to achieve the same or better ends sooner.
As an experienced CTO and CIO, I provide a potentially different perspective. I’ve read plenty of critiques by software engineers and architects who know the degree of error in their estimations and question the value from that angle. I hear a lot less from C-level executives for whom it may seem like there’s a wider array of reasons they need to demand software estimation from their teams.
Challenges with Software Estimation
First, let’s recap issues with software estimation from my previous posts and the many other articles critical of software estimation:
It is time that can be spent doing something else that is more valuable and aligned with making meaningful progress.
Delivers less value than assumed.
It is inaccurate and incomplete. People try to get better at estimation, and you can, but not enough to make it useful for what estimates are being ‘hired’ to do. People compensate by adding padding or being ‘lighter’ in the estimation approach - both are half-measures.
Decays quickly - i.e. starts inaccurate, and accuracy worsens quickly as we learn more.
Used as an arbitrary assessment of software development pace that does not correlate with actual pace.
Used to set deadlines and incentivise over-working.
Tries to serve many needs all at once, does none well.
Software estimation persists because:
It’s how people observe others approach the problem.
It feels like a direct response to questions from leadership, stakeholders and collaborators.
It feels like it solves multiple problems at once.
‘But, Software Estimation helps us clarify what needs to be done’.
One of the arguments for estimation I’ve heard is that it helps clarify our understanding of what needs to be done. And it’s true; it can, to a point. However, estimation does not help us learn how well our software will meet its aims. It can help you synthesise what you already know within your organisation but doesn’t help you understand how that will translate to the users. We can only learn about this by putting software into their hands; all else is conjecture or simulation.
It is undeniable that estimating software can translate into a better understanding of the work to be done. Anyone who has participated in some software estimation will have unearthed things like:
What the dependencies might be,
The sensible sequence of events,
Who needs to be involved,
What risks are there,
And many other aspects.
As the poor accuracy of estimations is hard to deny, it’s been common for software estimation proponents to make the case that a key benefit of software estimation is understanding the work better. That software estimation proponents hold this view is confirmed, given it's one of the more common defences of software estimation I’ve received for my previous posts on the topic:
But let’s suppose understanding the work better is the primary rationale for investing time into estimation. The time spent on estimation, which is considerable as it is often a group activity, could be spent on other practices or developing software. In that case, the more pertinent question should be:
Is estimation the best method to better understand what is required to develop the software successfully?
The answer to this question is no. There are better ways to understand the aspects I listed above. Using estimation because it appears to meet a range of needs may seem efficient, but it depends on what it is delaying to assess that. In this case, the time spent estimating software is delaying activities that are more likely to address the risks that jeopardise the developed software having the desired effects.
If I asked you to assess the dependencies that may need to be satisfied to release successfully, would your top answer be estimation? Would it be the top choice of activity to inform sequencing, identify stakeholders, identify risks, etc? Does this decision make sense if you are shortcutting the time available for other activities that may better achieve these ends?
This brings us back to the narrower range of needs software estimation might be suited to address. Coordinating with stakeholders such as go-to-market teams, sales, and customer service teams, as well as managing leadership expectations, are all needs that could benefit from software estimation. Then we arrive at a similar question:
Is estimation the best method to better understand what is required to coordinate software releases successfully?
The answer is also no. Forecasting based on meeting various desirable outcomes your software needs to succeed, or the rate of progress satisfying user stories, requirements or other markers of progress can provide much more accurate indicators of release readiness needed for successful coordination. Decoupling deployments from releases and involving collaborators and pilot user groups in the release process can reduce the dependencies and address release risks, addressing issues for which software estimation is often inappropriately used.
The cognitive flaw, which, in part, can lead people to believe software estimation provides more value than it does, is the appeal that it can solve more than one problem. I’ve covered the fallacy of grouping weaker rationales to make a stronger overall reason to do something, but that’s not how impact works.
The impact of action based on strong rationale almost always has an outsized effect compared to an aggregate of weaker reasons. You can liken it to the top items in a Zipf curve compared to anything, even a few more places down it.
There are exceptions, of course. I’ve led teams on a decent scale and in high-stakes environments with very little estimation of software effort occurring but still with the expectations of stakeholders actively managed by other means.
Occasionally, I would find cases where software estimation might be the simplest, lowest-effort answer to the question being asked. I am not arguing never to estimate; I am suggesting you be very specific about what problems you are trying to solve and then hire the best tool for the job. Avoid assuming that using a poor solution to multiple issues is more economical than a good singular solution to each problem you have to solve.
So, if software estimation is not the best approach in most cases, what are the alternatives?
Working smaller - slicing,
Forecasting,
Learning by doing,
Apply the most relevant practice to the problem at hand,
Move from how little we can spend to what we would be the most.
We will expand on what I mean by knowing what problem software estimation solves. We can then use this to assess whether software estimation is the best case in each scenario and consider alternati3ve options, such as those listed above, in the next post:
Do you use software estimation in your workplace? Who is the audience for the estimates? What are you ‘hiring’ software estimates to do? What are scenarios where software estimation is the best option? Share your experiences in the comments.