quantmleap

Scientific method refers to a body of techniques for investigating phenomena, acquiring new knowledge, or correcting and integrating previous knowledge. To be termed scientific, a method of inquiry must be based on gathering observable, empirical and measurable evidence subject to specific principles of reasoning. A scientific method consists of the collection of data through observation and experimentation, and the formulation and testing of hypotheses (source: http://en.wikipedia.org/wiki/Scientific_method).

If you are a regular reader of this blog you will notice that I take great care in substantiating my arguments (for better or worse) with collaborative evidence based on a scientific approach. Applying a scientific approach means that arguments can be independently verified, and supporting sources can be checked and their authenticity confirmed.

Last week I read two articles which made me further concerned about the need to realize and advocate a more scientific approach in regards to project management blogs, some of which are evidently based on gut-feel at levels that are below a reasonable threshold.

Lawrence M. Krauss, in a Scientific American article (Dec 2009 – titled “War Is Peace: Can Science Fight Media Disinformation?”) makes the observation that “The increasingly blatant nature of the nonsense uttered with impunity in public discourse is chilling. Our democratic society is imperiled as much by this as any other single threat, regardless of whether the origins of the nonsense are religious fanaticism, simple ignorance or personal gain.”

A similar note is raised by Jim Giles, in “Living in Denial: Unleashing a lie” (NewScientist.com – 21/05/2010). I encourage you to read Jim’s article as it is a fascinating tale of information (or more precisely dis) information management in the modern era. The key point arising from his article is the ease in which false data can propagate and promulgate to the point where fiction and reality are no longer indistinguishable.

The second article I read last week was published by Geoff Crane of Papercut Edge. In his article (titled “Annual Cost of Project Failure“) he makes a reference to a paper published by Roger Sessions in Nov 2009, titled “The IT Complexity Crisis: Danger and Opportunity“. Again, I encourage you to read Roger Sessions document as it is crucial for understanding the logical flaws in his approach. In a nutshell though, Roger Sessions makes the following assertions:

• A = 66% of all Federal IT dollars are invested in projects that are “at risk”;
• B = Let’s assume 65% of the above projects will fail;
• C = 2.75% = Proportion of GDP spent on IT
• D = 7.5 = a multiplier representing the total $ impact of a failed project on the economy
• E = $69,800 (USD Billion) – World Wide GDP
• Cost of failure = A x B x C x D x E = $6,180 (USD Billion).

I’ve read Roger Sessions paper and realized it heavily relies on data provided in the Budget of the United States Government, Fiscal Year 2009, Analytical Perspective. Before you run quickly to read this document, make sure you go straight to chapter 9, titled “Integrating Services with Information Technology” as this is the place where intellectual challenges associated with the above can be found.

As I was researching the above topic I’ve come across an excellent analysis done by Bruce Webster (see his article titled “The Sessions paper: an analytical critique“). The article provides excellent analytical explanation  outlining the methodological issues arising from Roger Session’s paper. I’m not going to repeat this here because I’d like to encourage you to read Bruce’s article.

The bottom line is that there is on-going state of confusion and misinformation regarding the current rate of IT projects’ failure. I’ve addressed in a number of previous posts (see Related Posts below) my reservations regarding the Standish report and its prolific interpretation. Now both Roger Sessions and Geoff Crane make the point that it is not the numbers that are important as much as the magnitude. Both fail to see that if the numbers are questionable, the magnitude is of no value what-so-ever.

I am seriously concerned, professionally, with the fact that further analysis and interpretation is carried out on the basis of shaky foundations, to the point where claims are taken as facts and these facts are further used to prove unsubstantiated assumptions. I suspect that some of the claims are self propelled by consultants who want to advance their services. Others by those who believe they simply come across a good idea to write an article about. At the end of the day though, regardless of the motives, it is the readers who need to make up their mind. The only way to allow our audience to make a proper judgement call is by providing properly presented, fairly substantiated information. Without such due diligence the words we write are not worth the 0′s and 1′s they are written on.

Related posts:

1. Projects failure rate – the conventional wisdom is wrong!
2. Projects failure rate – the sequel
3. Projects Failure Rate – the Threequel
4. The Availability Heuristic And the Projects Failure Rate Discussion
5. The Psychology Behind the Project Failure Debate