I have just finished reading an interesting post on HBR (“The Danger of Reading Something into Nothing” by Ted Cadsby) regarding the role randomness plays in our life. One of the key elements of the post, relevant also to the domain of project management, is the distinction that needs to be made and the awareness regarding the existence of Type I and Type II Errors.
Imagine the two scenarios:
- A jury, having evaluated and considered all the available information presented to it, declares an innocent person as being guilty.
- A jury, having evaluated and considered all the available information presented to it, declared a guilty person as being innocent.
And these two alternative scenarios:
- A doctor, having evaluated and considered all the available information presented to it, diagnoses a healthy person as being sick.
- A doctor, having evaluated and considered all the available information presented to it, diagnoses a sick person as being healthy.
See the pattern?
The professional language refers to example 1 as Type I Error and to example 2 and Type II Error. Alternatively, the two types are also called “False Positive” and “False Negative“.
At the heart of both errors lies the need to determine whether an hypothesis is correct or incorrect. In the jury’s case, it is an attempt to determine whether the hypothesis that the accused is guilty is correct (i.e. the accursed is indeed guilty) or incorrect (i.e. the accused is not guilty). In the medical case, it is an attempt to determine whether the hypothesis that the patient is sick is correct (i.e.e the patient is indeed sick) or incorrect (i.e. the patient is not sick).
The interesting point about both types of error is that they are evaluated against a (or contrary to the) state that it contrary to the expected natural state of the object under observation. In the case of the person on trial, the natural state of most people is that they are innocent. Similarly, most people would be naturally considered to be healthy. The natural state applied to an observation is called a null hypothesis, and by definition a null hypothesis can either be rejected or failed to reject.
Why “by definition”? Because in the case of the person going to have a medical health check, we cannot determine whether the person is healthy and we are limited to determining that he is sick (while he might really be healthy) or determining that he is healthy (while he might really be sick). Similarly, in the case of the person on trial, we cannot determine whether the person is innocent. All we can hope to do is decide that he is guilty (while he might really be innocent) or decide that he is innocent (while he might really be guilty).
The two types of errors are VERY MUCH relevant within the general context of project management and more specifically in the context of lesson learned at the end of the project.
Consider the following question:
The project which you have just finished managing has delivered is considered to be a [smashing success | terrible failure] – (pick one). You perform a post implementation review and identify areas where things were good and areas where improvement is required. Clearly the key assumption behind the exercise is the ability to determine cause and effect in order to ascertain the conditions which brought negative or positive results and quantify them into a set of lessons learnt for future projects.
And here in lies the challenge. You examine the successful parts of your project and quite likely assume that the actions you’ve taken in the context of these parts must have contributed positively to their success. Similarly, you examine the failed parts of your protect and quite likely assume that the action you’ve taken in the context of these parts must have contributed positively to their failure.
So when you assume that a successful delivery is a result of your action, you might be falling into both Type I and Type II Errors. Your success could have been achieved despite you making a serious error of judgement - Type I Error, or despite you taking a correct but incomplete action – Type II Error.
Roger Atkinson, in a 1999 article, posted in the International Journal of Project Management, argues that basing a decision on whether or not a project is successful on it meeting the cost, time and quality constraints is limiting as it does not take into account the possibility that other considerations are better (or equally) suited to determine whether or not a project is successful – a clear occurrence of a Type II Error. The possible bias resulting from this type of errors can more than explain the popular myth that the majority of project end in failure.
To use the point made by Ted Cadsby, “a deeper appreciation of randomness empowers us to ask the right questions when confronted with complex problems…Granted, anything that feels remotely akin to surrendering to randomness is hard for us: Our deep-seated need to feel in control motivates us to read something into nothing. But if managing complex problems were easy, they wouldn’t be complex, and we would still be living the straightforward but harsh lives of our progenitors.”
Seriously, think about it!