Comments on: The Scientific Method and the IT Projects Failure Rate debate http://quantmleap.com/blog/2010/05/the-scientific-method-and-the-it-projects-failure-rate-debate/?utm_source=rss&utm_medium=rss&utm_campaign=the-scientific-method-and-the-it-projects-failure-rate-debate Collection of thoughts about project management and other important things Sun, 05 Feb 2012 06:10:18 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Shim Marom http://quantmleap.com/blog/2010/05/the-scientific-method-and-the-it-projects-failure-rate-debate/#comment-2160 Shim Marom Thu, 09 Jun 2011 07:32:05 +0000 http://quantmleap.com/blog/?p=1271#comment-2160 <span class="topsy_trackback_comment"><span class="topsy_twitter_username"><span class="topsy_trackback_content">quantmleap: The Scientific Method and the IT Projects Failure Rate debate http://is.gd/gERFk9 #pmot #ftpm #pmp</span></span> quantmleap: The Scientific Method and the IT Projects Failure Rate debate http://is.gd/gERFk9 #pmot #ftpm #pmp

]]> By: shim_marom http://quantmleap.com/blog/2010/05/the-scientific-method-and-the-it-projects-failure-rate-debate/#comment-1287 shim_marom Thu, 16 Dec 2010 23:16:08 +0000 http://quantmleap.com/blog/?p=1271#comment-1287 Hi Karen, thanks for stopping by. The only <strong>credible </strong>research I've seen to date is the one published by Prof. Bent Flyvbjerg (see further details in http://quantmleap.com/blog/2010/11/bent-flyvbjergs-research-on-cost-overruns-in-transport-infrastructure-projects/). All others, although anecdotally interesting, lack any scientific rigour and should therefore, in my opinion, get ignored. I would appreciate your thoughts on this matter and any other observations you come up during your own research. Cheers, Shim. Hi Karen, thanks for stopping by.

The only credible research I’ve seen to date is the one published by Prof. Bent Flyvbjerg (see further details in http://quantmleap.com/blog/2010/11/bent-flyvbjergs-research-on-cost-overruns-in-transport-infrastructure-projects/). All others, although anecdotally interesting, lack any scientific rigour and should therefore, in my opinion, get ignored.

I would appreciate your thoughts on this matter and any other observations you come up during your own research.

Cheers, Shim.

]]> By: Karen http://quantmleap.com/blog/2010/05/the-scientific-method-and-the-it-projects-failure-rate-debate/#comment-1286 Karen Thu, 16 Dec 2010 22:22:31 +0000 http://quantmleap.com/blog/?p=1271#comment-1286 So, I was doing research on the failure rate of IT projects as well as change related projects. What are the most reliable figures you would stand by? So, I was doing research on the failure rate of IT projects as well as change related projects. What are the most reliable figures you would stand by?

]]> By: Can ‘Magical Thinking’ Mitigate Project Risks? | quantmleap http://quantmleap.com/blog/2010/05/the-scientific-method-and-the-it-projects-failure-rate-debate/#comment-710 Can ‘Magical Thinking’ Mitigate Project Risks? | quantmleap Sun, 24 Oct 2010 03:35:02 +0000 http://quantmleap.com/blog/?p=1271#comment-710 [...] In simple terms, Magical Thinking is about ignoring reality, or any reasonable application of a scientific method and putting ones faith in pure [...] [...] In simple terms, Magical Thinking is about ignoring reality, or any reasonable application of a scientific method and putting ones faith in pure [...]

]]> By: Shim Marom http://quantmleap.com/blog/2010/05/the-scientific-method-and-the-it-projects-failure-rate-debate/#comment-425 Shim Marom Fri, 16 Jul 2010 04:40:22 +0000 http://quantmleap.com/blog/?p=1271#comment-425 <span class="topsy_trackback_comment"><span class="topsy_twitter_username"><span class="topsy_trackback_content">quantmleap: The Scientific Method and the IT Projects Failure Rate debate http://is.gd/dtOF1 #pmot #ftpm</span></span> quantmleap: The Scientific Method and the IT Projects Failure Rate debate http://is.gd/dtOF1 #pmot #ftpm

]]> By: Scientific Method - MeMiM http://quantmleap.com/blog/2010/05/the-scientific-method-and-the-it-projects-failure-rate-debate/#comment-424 Scientific Method - MeMiM Tue, 15 Jun 2010 14:39:09 +0000 http://quantmleap.com/blog/?p=1271#comment-424 [...] offers both benefits and drawbacks to solving everyday or chaotic problems that go on in the fScientific Method - Utilizing the scientific method to solve a problem. admin --. by Elizabeth M. Young The [...] [...] offers both benefits and drawbacks to solving everyday or chaotic problems that go on in the fScientific Method – Utilizing the scientific method to solve a problem. admin –. by Elizabeth M. Young The [...]

]]> By: All Around the World News http://quantmleap.com/blog/2010/05/the-scientific-method-and-the-it-projects-failure-rate-debate/#comment-423 All Around the World News Tue, 15 Jun 2010 14:22:55 +0000 http://quantmleap.com/blog/?p=1271#comment-423 <strong>The Scientific Method and the IT Projects Failure Rate debate ......</strong> I found your entry interesting do I've added a Trackback to it on my weblog :)... The Scientific Method and the IT Projects Failure Rate debate ……

I found your entry interesting do I’ve added a Trackback to it on my weblog :) …

]]> By: Scientific Method | Ifdny http://quantmleap.com/blog/2010/05/the-scientific-method-and-the-it-projects-failure-rate-debate/#comment-422 Scientific Method | Ifdny Tue, 15 Jun 2010 14:15:59 +0000 http://quantmleap.com/blog/?p=1271#comment-422 [...] [...] [...] [...]

]]> By: Anonymous http://quantmleap.com/blog/2010/05/the-scientific-method-and-the-it-projects-failure-rate-debate/#comment-421 Anonymous Tue, 25 May 2010 23:19:12 +0000 http://quantmleap.com/blog/?p=1271#comment-421 Good on you mate. Appreciate your comment and even more so the links you provide as I haven't come across them before. I agree with you initial assertion that the worst offender is the Standish Group. In their report, which (as you correctly mention) there are substantial methodological issues, they have managed to create the illusion that all is wrong in IT project world. With this illusion extrapolated through cyberspace it is now almost impossible to stop the spread and other players are utlizing this as the foundation for further spreading the bad news. But no despair. The best way to combat disinformation is by providing proper, methodical and substantiated arguments. At the end of the day, the truth will prevail. Cheers, shim Good on you mate. Appreciate your comment and even more so the links you provide as I haven’t come across them before.

I agree with you initial assertion that the worst offender is the Standish Group. In their report, which (as you correctly mention) there are substantial methodological issues, they have managed to create the illusion that all is wrong in IT project world. With this illusion extrapolated through cyberspace it is now almost impossible to stop the spread and other players are utlizing this as the foundation for further spreading the bad news.

But no despair. The best way to combat disinformation is by providing proper, methodical and substantiated arguments. At the end of the day, the truth will prevail.

Cheers, shim

]]> By: svaucher http://quantmleap.com/blog/2010/05/the-scientific-method-and-the-it-projects-failure-rate-debate/#comment-420 svaucher Tue, 25 May 2010 22:45:06 +0000 http://quantmleap.com/blog/?p=1271#comment-420 The worst offender IMO is the Standish group. It tries to discourage the replication of their results (essential from a scientific point of view) [1]. Furthermore, there are significant methodological errors [2]. In a large study of project success/failure, the authors of [3] state: "It turns out that the Standish figures are not at all in accordance with reality.". They also state that the Standish figures tend to affect how organisations estimate effort (and that it is not a good idea). [1] http://doi.acm.org/10.1145/1145287.1145301 [2] http://doi.ieeecomputersociety.org/10.1109/MS.2009.154 [3] http://www.cs.vu.nl/~x/cone/cone.pdf The worst offender IMO is the Standish group. It tries to discourage the replication of their results (essential from a scientific point of view) [1]. Furthermore, there are significant methodological errors [2]. In a large study of project success/failure, the authors of [3] state:
“It turns out that the Standish figures are not at all in accordance with reality.”. They also state that the Standish figures tend to affect how organisations estimate effort (and that it is not a good idea).

[1] http://doi.acm.org/10.1145/1145287.1145301
[2] http://doi.ieeecomputersociety.org/10.1109/MS.2009.154
[3] http://www.cs.vu.nl/~x/cone/cone.pdf

]]>