I wrote in an earlier post (“how much process is too much process?“) about the tendency of some organizations to impose overly engineered processes in a (futile) attempt to increase quality.

Further to that and having been in the Information Technology sector for almost 30 years I can confidently and comfortably conclude that innovation, flexibility and grand availability of sophisticated and feature rich technologies have resulted in substantial increase to projects’ complexity with an increased risk of project integration failures.

Information Technology project tend to include, in one way or another, a number of database, networking, server, software packages and other perhaps more obscure technologies. Each of these technologies require the services of highly technical and trained personnel in order to best use the respective  tools to achieve the best result for the project. Now, the problem with today’s set of technologies is that they are all highly flexible, powerful, customizable and configurable; to the point that a considerable amount of effort need to be spent in order to ensure that the enormous capability built into the tool is harnessed appropriately.

Let’s use a common presentation tool as an analogy to explain the point above. Preparing a presentation is an easy task (provided you have the right tool and you know what you intend to convey to your audience). If you are an average user of Microsoft PowerPoint (or any other similar tool) you will surly know that in addition to the basic presentation capabilities you can introduce complex transitions, timers, sound and multimedia to enhance your presentation and turn it into an Oscar worthy event. Now, most people don’t require this extra elaborate capability and indeed, for many, this extra functionality is more of a distraction than an enabler as they get tempted by the available technology in an attempt to make use of all these gadgets because they are there.

A similar phenomenon which can be easily observed at the micro level can be seen also at the macro level. Organizations are lured into deploying and using large number of technologies, each of which come with a promise of amazing flexibility and functionality, while the consequences of deploying these technologies are not properly understood.

The questions I would like organizations to ask before deploying any new technology are:

1. Do we need this technology?
2. What tangible benefits, based on a clear selection criteria, does this technology have that we don’t already have today?
3. What percentage of the marketed feature would we actually use, compared with those features that we are really excited about but will never ever get used?
4. Assuming we conclude that we really, really, really need this technology, do we understand the impact on integrating this wonderful thing with our existing technologies?

The Conway’s Law which I got introduced to some time ago states that:

“…organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations“.

I would like to propose a slight variation on the above law (which I humbly call “Shim’s Law of the impact of Advanced technologies on projects’ risk“)

“organizations which design systems are constrained to produce designs which mirror the myriad of technologies the organization is invested in“.

Think about it!

I got intrigued by an article in LiveScience, titled “Angry Boss Can Bring Out Workers’ Creativity“.

The gist of the article is that, contrary to simple intuition, an angry boss, telling his/her employees off about their performance, may bring about as a consequence better performance and creativity. The key, however, is the level of engagement and desire to understand exhibited by the employee, a trait called epistemic motivation.

Epistemic Motivation, in simple words, relates to the level of involvement and dedication one has towards a particular area of involvement. It is less about the way you apply yourself physically towards a particular task as much at the level of intellectual and emotional involvement and your level of motivation to see that task done in the best possible way.

So here’s what the study shows. Study participants with a high degree of epistemic motivation reacted positively to angry feedback, resulting in more ideas, increased originality and breadth, and higher level of engagement. Those participants with low degree of epistemic motivation showed opposite results.

This and previous studies performed by the same research team conclude that for some employees (those exhibiting high degree of epistemic motivation) working with an angry boss would yield better results than working for a non-angry/happy one.

I personally have some methodological issues with the above study as although it demonstrates an increased level of performance as a result of angry reaction it does not necessarily imply that an angry boss is a better boss. I would suggest, and would await further research to confirm it, that a motivational, happy and easy going boss is likely to achieve the same increased performance from his/her team while, obviously, also appealing to those team members who, for what ever reason, do not show the same level of high epistemic motivation.

So, watch this space.

It took me some considerable time to realize that there are very few circumstances (which I can’t quite recall at the moment) where an objective view can be substantiated one way or another. There are quite a few excellent project management related bloggers out there, some I enjoy reading immensely; some I read knowing that their views will cause me a mild level of aggravation. The common thing I find across all these blogs is that in the main they haven’t got much in common as they all represent a particular set of circumstances, unique to the author’s environment.

 Let me try and explain.

 As project managers we all strive to do the ‘right thing’. The question any scientifically minded person is bound to ask would be what does the ‘right thing’ actually mean? There are number of Project Management related disciplines and guides (e.g. PMBOK, PRINCE) that attempt to define what the ‘right thing’ is. These ‘guides’ purport to provide a generic prescription detailing the type of things one ought to be doing in order to achieve a successful project delivery.

Quite a few project management bloggers attempt, in their regular blogs, to encourage the rest of us to execute the guidelines in the best possible way. Increase the effectiveness of our communication, implement earned value management, introduce better risk management processes, etc. Some bloggers go as far as suggesting (for example) that without the implementation of an effective earned value management in your project you risk blindly leading your project into unknown schedule and cost results. Similarly, if you don’t execute monte carlo simulation your planned schedule and cost are unreliable and useless.

 The truth of the matter is that although no two project managers operate in a similar environment; in the main it should be possible to distinguish between project managers who operate in a mature project management environment and those who don’t. Those who operate in a mature pm environment are right in their drive to stress the importance of thoroughly utilising high end project management processes (EVM, Monte Carlo, etc). For those operating in an immature project management environment, such calls are (1) irrelevant and (2) could be demoralizing as they serve as a reminder to areas of project management performance not currently in an implementable state.

 I haven’t got any credible statistics to justify what I’m about to say, and would be glad to be proven otherwise. I suspect that the majority (in fact the large majority) of project managers operate in sub optimal environments where key organizational processes required to implement the ‘right thing’ are simply not there. My view is that more blogging space should be allocated to supporting these project managers, than writing up posts for those operating at the upper end of the maturity scale.

 Our challenge is how to assist those project managers operating at the lower end of the maturity scale with practical, implementable, down to earth advice of how to achieve the best they can under the circumstances they’re in without preaching to them about their obligation to achieve a ‘right thing’ that we all know is simply tot possible to do.

Commitment

If you promise you will do something by a certain date and realize half way through that you can’t make it, please let me know. Don’t wait until the last minute as it will most likely affect other people and it takes time to reschedule material delivery and contractors’ time.

Communication

If I send you a message requesting information or assistance, don’t ignore it, I know you’re busy (I’m actually busy too) but there’s always time to be courteous and send a reply (any reply) – please manage my expectations and at least let me know when you’ll be able to respond with a proper reply

Attitude

Don’t assume that just because I’m the project manager it is “my problem”. The truth is that if you are part of the team then it is YOUR problem too. We’re all in it together and if the project fails it is a reflection on each and every one of us, not just ME.

Personality

Don’t be a bully; it doesn’t resonate well with me, or with others. Chances are you’ll get a much better response from others if you treat them with respect.

Time Manaement

Don’t come and see me every time you need information. It disrupts my schedule and forces me to divert my attention to you. If it is not urgent, send me an e-mail. I promise I will respond. If I’m very busy I will at least provide you with an indication when a more detailed response will be due.

Over my professional career I’ve worked with many project managers, some junior, and some senior. One character trait I’ve come to observe with all of them was that when things got tough they were always around to make sure the ship kept steering in the right direction. Whether large scale or small scale projects; massive impact or subtle impact; project managers where always there to project authority, control and sense of calmness over their project teams. In a sense, one of the duties a project manager often needs to perform is being a source of confidence and security for people to look up to and know that everything will be just fine.

 It is with the above in mind that I followed the investigation into the events that transpired around the Victorian bushfire disaster that occurred last year. In that context, one of the investigations’ findings was that the police commissioner at the time, Christine Nixon, did not do what any other project managers I know would (and should) have done, that is to stick around and ‘be visible’ to the troops. For those not familiar with the story (as probably most international readers won’t), Christine Nixon, while heading the Victorian Police, exhibited a ‘hands-off” style of leadership, and as fires raged through large parts of the state, went out for dinner at a North Melbourne pub.

 The investigation referred to Mrs. Nixon’s behaviours as demonstrating “error of judgement” – and damn right it is. Running a police force, or any other job of authority and control requires not just the skills but also the attitude. I can’t imagine a situation where a project team is battling issues, pressing deadlines etc. where a project manager would just leave and let his/her team battle on their own. Even if you can’t fix the code, complete the specifications or run the jobs, you can still walk around and provide words of encouragements, make coffee and deliver the pizza.

 So, when it comes to a police commissioner who decided that going out for dinner is more important than supporting her troops – I’m not impressed and neither should you.

One of the scheduling tools at the project manager’s disposal is the Critical Path Method, the outcome of which would be:

1. A list of all activities required to complete the project
2. The time (duration) that each activity will take
3. The dependencies between the activities.

With the above, the longest path of planned activities can be identified, and the “critical” activities (i.e those activities whose combined duration dictate how long the duration is planned to be) are identified.

There are a number of methodological issues with the Critical Path Method, one of which is that the view used to identified the critical path and the tasks it includes is a static view, correct at a point in time, and does not take into account the likelihood of any task on the project schedule taking longer (or shorter) than initially planned.

Let’s look at the following example:

Simply viewing at this schedule, without taking into account other supporting information could be misleading. One issue that could substantially affect the accuracy of this view is the availability (or lack) of resources (see elaboration on  this issue in my earlier post “The Critical Path Does not Tell You Everything You Need to Know About Your Project Constraints“). Another factor that needs to be examined is the level of comfort, or the likelihood of achieving the planned duration.

Having examined the above schedule, would you change your view of this project given the following additional information?

The table above introduces further information regarding the Optimistic, Pessimistic and Most Likely duration for each of these tasks. It clearly demonstrates that although tasks 2 – 5 share a Most Likely Duration, the level of risk associated with each is gradually increasing, from 5 days in task1 to 9 days in task 5.

Utilizing the above added information to better understand the risk dimension associated with the schedule requires the execution of a sensitivity analysis.

Sensitivity Analysis is defined as a “Simulation analysis in which key quantitative assumptions and computations (underlying a decision, estimate, or project) are changed systematically to assess their effect on the final outcome. Employed commonly in evaluation of the overall risk or in identification of critical factors, it attempts to predict alternative outcomes of the same course of action. In comparison, contingency analysis uses qualitative assumptions to paint different scenarios. Also called what-if analysis“.

In other words, a greater insight into the risks hidden behind the project schedule can be derived by adjusting the various tasks’ durations (within their PERT parameters) and confirming, for each such variation, whether a task is included or excluded from the critical path. Performing such adjustments a large number of times would result in an indication of how many times, or in what percentage of times, did any particular task appear on the critical path. The larger the frequency, the greater the need to examine the task and put in place the mechanism to ensure that the task does not negatively deviate from its scheduled duration.

Just to wrap this up, let’s look at the schedule from two additional, extreme perspectives:

The following is a Critical Path view of the project schedule should all tasks durations happen to match the best case duration estimates:

Next is the view of the project schedule should all tasks durations happen to match the worst case duration estimates:

Performing Sensitivity Analysis on the above project would result in a result similar to the one outlined below:

What the above diagram suggests is that when running a simulation with a large number of samples (lets say 1,000 times), then Task1 will appear on the Critical Path 100%, Task4 and Task5 – 39% of the times, etc.

Wouldn’t this be useful to know?

A 2007 article by Young Hoon Kwak and Lisa Ingall, titled “Exploring Monte Carlo Simulation Applications for Project Management”, examines the Monte Carlo Simulation method and its uses in the field of Project Management.

Apart from being a good reference document, where a brief history of this technique is being discussed and explained, this article provides a good review of various studies published around the benefits as well as the potential complexities associated with implementing this technique in real life situations.

The article points out that one of the limitations of using this technique as being project managers’ discomfort with statistical approaches, as well as lack of thorough understanding of the method (see also my earlier post discussing  this issue in “Some Risk Management Related Thoughts“.

Discussing the process for utilizing Monte Carlo Simulation in the context of Time Management, the article suggests the following steps are commonly used:

1. Utilize subject matter expertise to assign a probability distribution function of duration to each task or group of tasks in the project network;
2. Possibly use Three-Point-Estimates to simplify this process, where an expert knowledge is used to supply the most-likely, worst-case and best-case durations for each task or group of tasks;
3. Fit the above estimates to a duration probability distribution (such as Normal, Beta, Triangular, etc.) for the task;
4. Execute the simulation and use the results to formulate expected completion date and required schedule reserve for the project.

Outlining the advantages of utilizing Monte Carlo Simulation applications in Project Management, the article points out that its primary advantage is in being an “extremely useful tool when trying to understand and quantify the potential effects of uncertainty in projects“. Clearly, not utilizing this technique, project managers lack a powerful tool that can result in not meeting the project’s schedule and cost targets. Better quantification of the necessary schedule and cost reserves can substantially reduce such risks.

The article highlights the importance of having access to expert knowledge and prior experience and detailed data from previous projects in order to mitigate the inherent issues of estimates uncertainly which would ultimately affect the quality of the simulation results. This is correct not just with respect to the three-point-estimates but also with respect to choosing the correct probability distributions with which to model these estimates.

An interesting point is raised when referring to an earlier study published  by Grave R. (2001. “Open and Closed: The Monte Carlo Model,” PM Network, vol. 15, no. 2, pp. 48-52) which discusses the merits of using different types of probability distributions for project task duration estimates. Grave suggests the use of open-ended distribution (the lognormal distribution) instead of using closed-ended distributions (such as the triangular distribution) when performing the Monte Carlo Simulations.

His logic is as follows: A closed-ended distribution (e.g. triangular distribution) does not consider the possibility  of a task duration completing BEFORE the best-case or AFTER the worst-case duration estimate. However, in real life projects, due to various constraints, it is possible for a task to complete before or after the best or worst scenarios.

When an open-ended distribution is used, the possibility for exceeding the upper limit of the task duration is recognized, thus making the simulation more realistic.

The article touches (although very briefly) on one of the aspects used within the context of Monte Carlo Simulation which is the Criticality Index (I will endeavor to provide a more detailed discussion about  this feature in a future post). In a nutshell, the Criticality Index is a reflection of the rate at which the task appears on the critical path of the project through the simulation iterations.

Overall an interesting article. If you are already using Monte Carlo Simulation as part of your portfolio of project tools, this article will encourage you to keep doing so. And if you’re not – you’ll need to ask yourself – WHY?

Want to know more what this thing is all about, check for more info in my “Monte Carlo Simulation – Project Risk Analysis Report”

Ok, I’m having a bit of a fun with this one.

Submit a copy of your project plan (in Microsoft Project format only at the moment) and I will send back to you a high level risk assessment of your project schedule’s based on a Monte Carlo Simulation.

There’s absolutely no catch. The reason I’m doing it is because it seems to me that many don’t yet understand the benefits of using this technique to better understand the risks that lie dormant within their project schedules.

If you are interested in this FREE offer fill in and submit the details below.

And by the way, I give you my guarantee that once I process your project schedule I will not make any further use of it and it will be erased/discarded and will not be used for any purpose other than attempting to provide you with this service.

To learn / read more about this topic check out the Related Posts listed below.

Over a week ago I came up with an offer to provide readers with the unique opportunity to run their project plans through a Monte Carlo Simulation and provide them with a high level summary of the results.

The response exceeded my expectations and up until now I was approached by 30 readers to provide them with an assessment of their plan.

Having gone through this experience I thought it would be appropriate to summarize my findings from this exercise, without, obviously, revealing the individual nature of the plans that have come under my hands.

All plans provided were in a Microsoft Project format and none was larger than 500 lines. To my surprise, none of the plans provided had the basic scheduling distribution populated (Most Likely, Optimistic and Pessimistic estimates) for each of the tasks. In order to overcome this issue I have applied an across the board triangular distribution rule of 75%, 100%, 125% (i.e. assuming that the Optimistic Estimate will also be 75% of the Most Likely Estimate and that the Pessimistic Estimate will always be taken as 125% of the Most Likely Estimate).

Not surprisingly and as expected, in all cases, the 80% likelihood of finishing the project on or before a certain date was well after the plans’ deterministic date (i.e. the date predicted by the software as being the project’s completion date).

Not all project schedules provided had costing information provided but in those who had an expected project cost, this as well has shown a deterministic cost well below the 80% likelihood mark.

As I’m excited by what I’ve seen I intend to run with this activity for few more days so the opportunity to use this option is still open (but not for much longer).

To ensure the successful completion of a project, it is of utmost importance for the project manager to find ways to handle uncertainties that can pose potential risks for a project. Risk management is an iterative process. Risks can relate to any aspect of the project – be it the cost, schedule, or quality. The key to managing risks is to identify them early on in the project and develop an appropriate risk response plan.

To develop a Risk Response Plan, you need to quantify the impact of risks on the project. This process is known as quantitative risk analysis wherein risks are categorized as high or low priority risks depending on the quantum of their impact on the project. The Project Management Body of Knowledge (PMBOK) advocates the use of Monte Carlo analysis for performing quantitative risk analysis.

What is Monte Carlo Analysis?

Monte Carlo analysis involves determining the impact of the identified risks by running simulations to identify the range of possible outcomes for a number of scenarios. A random sampling is performed by using uncertain risk variable inputs to generate the range of outcomes with a confidence measure for each outcome. This is typically done by establishing a mathematical model and then running simulations using this model to estimate the impact of project risks. This technique helps in forecasting the likely outcome of an event and thereby helps in making informed project decisions.

While managing a project, you would have faced numerous situations where you have a list of potential risks for the project, but you have no clue of their possible impact on the project. To solve this problem, you can consider the worst-case scenario by summing up the maximum expected values for all the variables. Similarly, you can calculate the best-case scenario. You can now use the Monte Carlo analysis and run simulations to generate the most likely outcome for the event. In most situations, you will come across a bell-shaped normal distribution pattern for the possible outcomes.

Let us try to understand this with the help of an example. Suppose you are managing a project involving creation of an eLearning module. The creation of the eLearning module comprises of three tasks: writing content, creating graphics, and integrating the multimedia elements. Based on prior experience or other expert knowledge, you determine the best case, most-likely, and worst-case estimates for each of these activities as given below:

The Monte Carlo simulation randomly selects the input values for the different tasks to generate the possible outcomes. Let us assume that the simulation is run 500 times. From the above table, we can see that the project can be completed anywhere between 11 to 23 days. When the Monte Carlo simulation runs are performed, we can analyse the percentage of times each duration outcome between 11 and 23 is obtained. The following table depicts the outcome of a possible Monte Carlo simulation:

This can be shown graphically in the following manner:

What the above table and chart suggest is, for example, that the likelihood of completing the project in 17 days or less is 33%. Similarly, the likelihood of completing the project in 19 days or less is 88%, etc. Note the importance of verifying the possibility of completing the project in 17 days, as this, according to the Most Likely estimates, was the time you would expect the project to take. Given the above analysis, it looks much more likely that the project will end up taking anywhere between 19 – 20 days.

Benefits of Using Monte Carlo Analysis

Whenever you face a complex estimation or forecasting situation that involves a high degree of complexity and uncertainty, it is best advised to use the Monte Carlo simulation to analyze the likelihood of meeting your objectives, given your project risk factors, as determined by your schedule risk profile. It is very effective as it is based on evaluation of data numerically and there is no guesswork involved. The key benefits of using the Monte Carlo analysis are listed below:

• It is an easy method for arriving at the likely outcome for an uncertain event and an associated confidence limit for the outcome. The only pre-requisites are that you should identify the range limits and the correlation with other variables.
  
• It is a useful technique for easing decision-making based on numerical data to back your decision.
  
• Monte Carlo simulations are typically useful while analyzing cost and schedule. With the help of the Monte Carlo analysis, you can add the cost and schedule risk event to your forecasting model with a greater level of confidence.
  
• You can also use the Monte Carlo analysis to find the likelihood of meeting your project milestones and intermediate goals.
  

Now that you are aware of the Monte Carlo analysis and its benefits, let us look at the steps that need to be performed while analysing data using the Monte Carlo simulation.

Monte Carlo Analysis: Steps

The series of steps followed in the Monte Carlo analysis are listed below:

1. Identify the key project risk variables.
   
2. Identify the range limits for these project variables.
   
3. Specify probability weights for this range of values.
   
4. Establish the relationships for the correlated variables.
   
5. Perform simulation runs based on the identified variables and the correlations.
   
6. Statistically analyze the results of the simulation run.
   

Each of the above listed steps of the Monte Carlo simulation is detailed below:

1. Identification of the key project risk variables: A risk variable is a parameter which is critical to the success of the project and a slight variation in its outcome might have a negative impact on the project. The project risk variables are typically isolated using the sensitivity and uncertainty analysis.
   

   Sensitivity analysis is used for determining the most critical variables in a project. To identify the most critical variables in the project, all the variables are subjected to a fixed deviation and the outcome is analysed. The variables that have the greatest impact on the outcome of the project are isolated as the key project risk variables. However, sensitivity analysis in itself might give some misleading results as it does not take into consideration the realistic nature of the projected change on a specific variable. Therefore it is important to perform uncertainty analysis in conjunction with the sensitivity analysis.
   

   Uncertainty analysis involves establishing the suitability of a result and it helps in verifying the fitness or validity of a particular variable. A project variable causing high impact on the overall project might be insignificant if the probability of its occurrence is extremely low. Therefore it is important to perform uncertainty analysis.
   

2. Identification of the range limits for the project variables: This process involves defining the maximum and minimum values for each identified project risk variable. If you have historical data available with you, this can be an easier task. You simply need to organize the available data in the form of a frequency distribution by grouping the number of occurrences at consecutive value intervals. In situations where you do not have exhaustive historical data, you need to rely on expert judgement to determine the most likely values.
   
3. Specification of probability weights for the established range of values: The next step involves allocating the probability of occurrence for the project risk variable. To do so, multi-value probability distributions are deployed. Some commonly used probability distributions for analyzing risks are normal distribution, uniform distribution, triangular distribution, and step distribution. The normal, uniform, and triangular distributions are even distributions and establish the probability symmetrically within the defined range with varying concentration towards the centre. Various types of commonly used probability distributions are depicted in the diagrams below:
   

   
   

   
   

4. Establishing the relationships for the correlated variables: The next step involves defining the correlation between the project risk variables. Correlation is the relationship between two or more variables wherein a change in one variable induces a simultaneous change in the other. In the Monte Carlo simulation, input values for the project risk variables are randomly selected to execute the simulation runs. Therefore, if certain risk variable inputs are generated that violate the correlation between the variables, the output is likely to be off the expected value. It is therefore very important to establish the correlation between variables and then accordingly apply constraints to the simulation runs to ensure that the random selection of the inputs does not violate the defined correlation. This is done by specifying a correlation coefficient that defines the relationship between two or more variables. When the simulation rounds are performed by the computer, the specification of a correlation coefficient ensures that the relationship specified is adhered to without any violations.
   
5. Performing Simulation Runs: The next step involves performing simulation runs. This is typically done using a simulation software and ideally 500 – 1000 simulation runs constitute a good sample size. While executing the simulation runs, random values of risk variables are selected with the specified probability distribution and correlations.
   
6. Statistical Analysis of the Simulation Results: Each simulation run represents the probability of occurrence of a risk event. A cumulative probability distribution of all the simulation runs is plotted and it can be used to interpret the probability for the result of the project being above or below a specific value. This cumulative probability distribution can be used to assess the overall project risk.
   

Summary

Monte Carlo simulation is a valuable technique for analyzing risks, specifically those related to cost and schedule. The fact that it is based on numeric data gathered by running multiple simulations adds even greater value to this technique. It also helps in removing any kind of project bias regarding the selection of alternatives while planning for risks. While running the Monte Carlo simulation, it is advisable to seek active participation of the key project decision-makers and stakeholders, specifically while agreeing on the range values of the project risk variables and the probability distribution patterns to be used. This will go a long way in building stakeholder confidence in your overall risk-handling capability for the project. Moreover, this serves as a good opportunity to make them aware of the entire risk management planning being done for the project.

Though there are numerous benefits of the Monte Carlo simulation, the reliability of the outputs depends on the accuracy of the range values and the correlation patterns, if any, that you have specified during the simulation. Therefore, you should practice extreme caution while identifying the correlations and specifying the range values. Else, the entire effort will go waste and you will not get accurate results.

Want more? Check out my special offer here.