The raging debate surrounding the #NoEstimates concept has left me with a sense of frustration one usually feels when coming across an unfinished business. While a number of individuals on both sides of the debate have attempted to address the key differences of opinion, the question still seems to be unresolved with each side sticking to their guns, unable to agree on a common ground or a framework of understanding in which all parties can feel vindicated. In this post I will attempt to outline the key arguments for and against #NoEstimates. When possible and if appropriate I will quote these arguments verbatim, sourcing them from a number of blogs where this topic has been discussed (sometimes indirectly) extensively. I will provide links to these quotes but do my utmost to ensure I do not take these quotes out of context.
The format I plan to adopt for this post is as follows. Each argument, in favor or against the #NoEstimates proposition, will be listed and a statement in support and against the proposition will be provided. whenever appropriate I will add further commentary where I will attempt to narrow the gap and articulate the assumptions arising from each or both arguments.
As the end of this post I hope you will have a better appreciation of the arguments and you will gain the ability to articulate your own answer to the following questions:
- Are cost estimates required in order to manage software projects?
- Are cost estimates an effective tool for controlling costs?
- Do estimates stifle creativity and kill innovation?
- Do people need estimates? And if so, why?
- Is the very act of estimation results in the creation of uncertainty?
- Is estimation a practice still hanging over from the Waterfall era?
- Is No Estimation better than Bad Estimation?
- Is Estimation really just a form of Guessing?
- Are estimates necessary for Governance? Is it reasonable to require estimates for the purpose of pacifying governance needs?
- Is there any point in providing estimates when it is known that many projects fail due to lack of credible estimates? And aren’t estimates a tool used to apportion blame afterwards anyway?
- Is costing a necessary tool for determining business value?
- Does the scope drive the budget or does the budget drive the cost?
- If estimates are required in order to support decisions, can decisions be made without estimates?
We have a lot to cover so let’s make a start!
1. Making software development decisions based on estimates are counter intuitive
#NE – There is a tendency in software development to, counter intuitively, take a reverse approach to budgeting we would employ when we make purchasing decisions in our lives. Instead of deciding our budget based on how much we have available, or are willing to spend – as we would personally do – we decide it using an estimate from the supplier of how much they tell us the software that we want will cost. So, while we have a pretty good appreciation of how much much we have, want or willing to spend, we are unnecessarily engaged in a bidding process with the view to find the cheapest bidder in order to save money and squeeze as much as we can for our real budget. In such case, would we not gain a better outcome if we work iteratively and collaboratively with the supplier based on our budgetary and / or time constraints? (link).
#E – The claim that iterative and collaborative work can be used as a substitute for up-front estimation of cost is challenged by Mike Cottmeyer in “Should You Use Agile to Build Your Next Home?“. The gist of his argument is that most people when they are spending their own money, want to have some idea of what they are going to get when the time and money run out. They want some assurance that they’ll have enough of the product they have ordered to make the whole investment worthwhile. Suggesting an incremental delivery, while appealing in theory, is not necessarily a practical outcome as even a fully functional software, when lacking the greater context might be unusable (just like having a fully functional kitchen might not be usable unless the bedrooms and the other amenities have also been complete).
The other factor worth mentioning here is the behavioral aspect driven through our understanding of economic theories. The concepts of Scarcity, Opportunity Costs and Cost Benefit Analysis dictate that people, intuitively, would look to determine future value before they make an investment decision. This determination is done, among other methods, by comparing one perceived outcome against other alternatives, and selecting the one that maximizes the perceived value (or Return on Investment).
2. Estimates are seldom correct so why use them?
#NE - A casual search on Google will reveal that faulty or wrong estimation is seen as one of the leading reasons for projects’ failures. Proponents of the #NoEstimates proposition raise the following arguments regarding the lack of credibility associated with the provision of estimates:
- Estimates are often gamed, and for various reasons (link)
- They are often based on zero to low knowledge (link)
- They are often being plucked out of thin air (link)
- Time and cost estimates are based on an up-front estimate of scope (i.e. a guess) and as such are guesses at best, leading to obvious dysfunctions like death-marches, low quality, etc. (link)
- Asking teams to estimate how long their work will take has connotations that their output is being measured by an external party (management), creating an environment of fear and massaging figures to reflect what is desired rather than what i predicted (link)
- We should’t be defining all our scope up-front, meaning we shouldn’t estimate all our scope up-front, meaning we shouldn’t be defining our delivery date based on our scope (link)
- It seems like we keep failing at getting good at estimates, even though we are working hard at becoming better at doing so, then maybe we are expecting estimates to do something for us that just can’t be done. (link)
- Estimates, quite often, are a result of a a wild guess, aimed at providing some numbers that seem plausible and will allow the decision makers to approve the start of a project with a clear conscience. That is, when things sour later on, they can always blame the developers for giving them inaccurate estimates. And the developers can always blame the ‘requirements people’ for giving them unclear, incorrect, or incomplete requirements. And the requirements people can always blame the stakeholders for providing bad direction about what was important. And so on. Regardless of who is part of this process, it is one big happy circle of blame that lets us all do the wrong thing and still keep our jobs… (link)
- When estimating a date or cost you are creating uncertainty around those things, because you are guessing. You are saying “we’ll deliver somewhere between here and here”. However, if your delivery date and/or cost is set by a real constraint, as advocated by the #NoEstimates approach, you have created certainty around those things. (link)
- And see also Jens Schauder’s ‘8 Reasons why Estimates are too low‘ and Glen Alleman’s specific response.
#E – The arguments against estimates can be classified into two categories; practical and behavioral.
Lets’ start with the practical considerations first:
‘Doing’ estimates right (though not necessarily correct) requires effort. While we are all accustomed to performing estimates in most facets of our lives (when we drive our car, rush to the station to catch the train, do our shopping at the supermarket, etc) there are instances or circumstances in which making the wrong estimate can have more severe consequences. If we try to cross the street and misjudge the distance and speed of an approaching car we could be involved in a fatal accident. If we misjudge the effort required to develop a software feature we could end up being responsible for a business loss that could affect our very livelihood.
When we make an estimate we always take a risk. When we buy one bottle of milk for our daily morning coffee we base our purchasing decision on past experience coupled with some probability assessment regarding the likelihood of past patterns breaking up in the future. While we don’t explicitly enunciate that level of uncertainty, we subconsciously are aware of it. We know, intuitively, that it is never 100% but are happy to concede that there is X% probability (perhaps 90%) that one bottle of milk would be sufficient to meet our needs for the next few days. Had the probability been low, we would most likely buy two bottles.
While making decisions (based on estimates) come naturally to us when we cross the road, the reason why we are comfortable making these probability assessments is because we have learned to execute them based on past experience. From an early childhood our parents have taken our hand and instilled in us the need to accumulate experience and knowledge we could utilize at a later age, without giving it much thought. Statisticians and Economists would call this process Reference Class Forecasting (and see Kailash Awati’s elaboration on this topic in Improving Project Forecasts).
In the world of software development we are asked to exhibit the same assessment skills with the appreciation that our reference class might not be obvious up front – i.e we might be having a Reference Class Problem. In a nutshell, this problem “…arises when we want to assign a probability to a proposition (or sentence, or event) X, which may be classified in various ways, yet its probability can change depending on how it is classified ” (from The Reference Class Problem is Your Problem Too, in Kailash Awati’s The reference class problem and its implications for project management).
At the practical level, being dismissive of the validity of estimation could be an attempt at avoiding the need to establish a reference class. If you don’t have a reference class on which to base your estimates then (as often argued by Glen Alleman) build one, run a pilot project, execute an experimental iteration, ask others who’ve done something similar. Do something to reduce the uncertainty and increase the certainty. There should be no excuses for not trying to establish some level of reference class baseline and progressing based on that starting point.
And now to the behavioral considerations…
The notions of gaming the estimates, plucking figures out of thin air, perceiving estimates as a mean to establish a management control and associating estimates with future apportioning of blame are all symptoms of a dysfunctional organizations. The business dilemma we need to address is whether the elimination of estimates would turn the organization into a functional one or would that just be a way of dealing with the symptom while ignoring the real problem. A better proposition should be to suggest methods to change the organizational behavior such that these attitudes are transformed into more suitable ones. Implementing the practical practices established for the delivery of estimates can go someway towards ‘fixing’ the organization. If you are a strong and vocal proponent of #NoEstimates you could equally be in a position to strongly support calls for taking the necessary steps needed to change the organizational attitude such that developers would not feel under siege every time an estimate is required (and see also the discussion here re. dysfunctional organizations).
3. Estimation = Waterfall
#E – See Mike Cottmeyer in “Should You Use Agile to Build Your Next Home?“. Tying estimates with Waterfall is not a solid argument and it seems to appeal to those with emotional associations tied to the Agile vs Waterfall metaphorical divide.
4. Why do customers need estimates?
#NE – The main reason we do estimates is because people need to make decisions. This means that the key requirement is for people to be able to make decisions. It is true that estimates are often used to help us make decisions. But estimates are not the only way and for many of the decisions we need to make they are likely not the best way to go. One good way to make decisions is by adopting the “lean approach”, by learning from our mistakes and by frequently taking corrective action (link).
#E – There is no disagreement about the conjecture that estimates are required to make decisions. As highlighted earlier, whether or not we are engaged in formal and conscious estimation process – we are still estimating sub-consciously. And we do it because this is how we intuitively engage in risk management. We are always on the look for signals of danger and for opportunities. And while we are engaged in this type of behavior in every facet of our lives, we should more inclined to exhibit such behavior when spending other people’s money. The type of decisions we need to be able to ethically make are about whether we ‘buy’ this software or ‘make’ it. Do we invest the money in this product or in any one of a number of other opportunities.
The decision regarding the investment path to be taken cannot be solely based on the perceived business value, as while a number of development opportunities might present a value proposition, the value differentiation might not be so clear as to determining the one we ought to pick for further development. And this is where cost becomes a deciding factor. It is called cost-benefit-analysis and it is a sensible risk management strategy.
Some final notes
The discussion surrounding the #NoEstimates proposition is one we need to have. The issues highlighted here are not raised with the intention to keep people from exploring and challenging established assumptions but rather as a concern against applying judgement without considering the context and landscape within which these judgments ought to be made.
No doubt #NoEstimates could work in environments where such a proposition will be accepted by the stakeholders who’s money is put on the line. While I have not yet had the opportunity to operate in such an environment does not mean that such environment does not exist. And this is exactly the point where context need to be injected into the discourse around this issue. It is about establishing known boundaries or parameters within which #NoEstimates can (and perhaps also ought to) work. Some mention areas of R&D as suitable for this and perhaps this is correct. Establish a budget you are willing to use for experimentation (based on your risk appetite) and let the team iterate until such time that you either run out of money, run out of risk appetite or hit the bull eye. Other areas could be small developments with low risk. The domain is known and the time taken to produce formal estimates is better get used on actual development. There is, however, a need to avoid broad categorizations that add little to no credibility to the argument and add a sense of religious ideology to a domain that requires no such similarities.
Think about it!