quantmleap

Jack Shafer published on 24th March an interesting post in the Slate online magazine, titled “Numbers Are Hard To Come By“. The premise of his article is that journalists, not completely informed or updated with quantitative and qualitative data relating to their area of reporting, are not forthcoming in their public reporting about that lack of data and are not shying away from making unwarranted and unsubstantiated conclusions.

As I was contemplating the unscrupulous nature of some journalists I couldn’t ignore the lingering thought that similar, though perhaps less consequential, behaviour is also prevalent in project management blog-land.

Few examples come to mind:

Earlier discussions over emphasising the role that Social Media tools play in the context of project management. It is obvious to me that these were motivated more by wishful  thinking than clear evidence.

In a similar context, the public debate abut PM 2.0 – yet again – a debate about something that ended up being absolutely nothing.

Agile as the next saviour of IT projects – yet another example to a debate based entirely on expectations and wishful thinking – without any credible evidence to back it up.

What I find particularly fascinating is the fact that discussions about such topics can carry on and flare up for long periods of time, with none of the participants in this discussion having a clue about where reality ends and speculation begins. As Jack Shafer suggests in his article, there is nothing wrong with speculating and making calculated guesses, provided that such inferences are accompanied with a suitable disclaimer, noting the lack of data and providing the methodological context for making the assumption. But, as Jack Shafer concludes “I’ve yet to see a reporter do this, if only because it’s too easy to build an audience-pleasing story on soft numbers“.

Think about it!

Given the recent influx of articles about Agile, particularly given the recent decision made by the PMI to introduce an Agile PM Certification, I couldn’t escape the thought that we’ve been here before. Different players, different scenery, even different name, but the same act, all over again.

There is something fascinating about us, humans, where – given our natural discomfort with uncertainty – we cling to statements projecting certainty, even when these statements are not supported by evidence. Cast your mind back to the proliferation of buzz words and other claims from recent times, “PM 2.0”, “Social Media”, “Standish Report”, “Upsizing”, “Downsizing”, “Rightsizing”, etc. The consulting industry is thriving on the creation and propagation of such terms as this gives its members the public clout they require in order to promote their expensive services.

This brings to mind the following two quotes:

“Science is a process of separating the demonstrably false, from the probably true” (Michael Zimmerman)

“The much hyped leaders debates are just a political black hole that consume time, money, energy, resources, staff and adrenalin, and rarely produce anything other than a mush tie  allowing all the spin doctors to claim a phony victory” (Rod Love)

To have a serious debate we need more than emotions and sentiments, we need credible data on which we can make inferences and policy decisions.  In the context of the Agile debate that has yet to happen.

Think about it!

One of my favourite books (and films) is “The Lord of the Rings“. There are many situations presented in this book which make me pause, think and contemplate. Over the years I have joined Bilbo and the fellowship of the ring on their journey to Mount Doom; and on each one of these journeys I identified yet another existential point worth considering that I was unaware of, or acutely conscious of, before.

J.R.R. Tolkien’s trilogy offers a substantial number of unforgettable quotes, two of which I find particularly relevant to a trend I believe we all witness in recent days.

Things are now in motion that cannot now be undone.
–Gandalf–

The age of man is over. The time of the Orc has come.
–Orc commander–

The trend I am referring to is the seemingly growing acceptance of Agile as a mature, ready to enter the centre stage, methodology; one that can take project executions into the next stage of maturity. The premise of this acceptance is that given the known deficiencies in current software development projects, adoption of Agile techniques and methodologies could, and indeed will, result in greater numbers of projects actually crossing the finish line successfully.

The above is a big call to make, one which I personally can’t bring my self to make.

In an earlier post (“Project Management RAD Certification Anyone?“) I’ve made the non-judgemental observation that there seems to be a positive correlation between the increase in the ratio of Generation Y in the workforce, and the apparent drive towards Agile solutions. I have assumed that the correlation is self-explanatory, but having discussed this further with colleagues and friends I now realize that certain level of elaboration is required.

Now, I know the correlation identified above is circumstantial at best and further research is required in order to give it a more scientific credibility. But I suspect however that it is good enough as a starting point. What the above is attempting to demonstrate is that there seems to be positive affinity between the fundamental concepts behind the Agile methodology and the driving attributes of Generation Y.

If the above proposition is correct then the following could be concluded:

1. The move to Agile is not a temporary glitch (or ‘correction’ – a term used in financial markets) but rather a sustained and lasting one.
2. Like the revolutions in the Middle East, the call for applying Agile is now expected to come ‘bottom up’ and not ‘top down’.
3. The time will finally come, and soon, where we would be able to look back and give Agile a proper enterprise adaptability score – i.e. does it have what it takes to deliver enterprise solutions.

Irrespective of whether we think Agile is ready for prime time TV (or alternatively – are WE ready), we can look forward to interesting times ahead.

Think about it!

I still can’t get over the recent decision by the PMI to introduce yet another, Agile PM, certification.

It is not that I reject the concept of certifications. I am a PMP and I respect the value proposition that such certifications project. What I am concerned about is the perception that Agile is the solution to our ailing software development projects. Not that I under value the contribution that Agile has made to software development, but, let’s be honest about it, with Agile or without it, we are still some way off the yellow brick road.

In the mid 1990′s I worked on software projects that used the RAD approach. RAD, as of Rapid Application Development, was developed in 1991 by James Martin (the guru of Information Engineering), and introduced the concept of iterative development through the introduction of quick prototypes; heavily predicated on constant and close involvement of relevant business owners.

In the mid 1990′s no one was cynical enough to suggest that a whole project management approach needs to be developed and wrapped around RAD. It was understood that RAD is just one of a number of software development approaches, and as such, can be implemented within traditional project management disciplines.

So why are things different in 2011?

Ok, I’m going to be slightly wild here, and highly speculative.

The apparent adoption of Agile principles seems to coincide with the increased ratio of Gen ‘X’ and ‘Y’ in the work force. It is not difficult to see why, especially for our colleagues who are in the Gen ‘Y’ classification, the social and collaborative aspects of Agile will seem like a natural fit.

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

Now let this the above statements permeate for a second in your mind, and answer the following question: What Generation bears the prejudicial personality most likely to grasp and develop an affiliation with the statements above.

• Baby Boomers? – Unlikely!
• Generation X? – Yes, but only partially
• Generation Y?- Oh Yes!!!

Without conducting any research, and I’m afraid the supporting data is not (yet) there, I’m happy to conclude that the ‘move’ to Agile is a reflection of our time. Gen ‘Y’ is known to exhibit strong social-media and collaborative needs. This is not a matter of good or bad, it is rather a sociological observation. And as we re-think and digest this observation we need to remember that this phenomena on its own, is not a good enough reason to go the Agile way. Like the RAD of the 1990′s, the decision for adopting a methodology needs to be done based on its objective merits, not based on the whims of those proposing it.

Think about it!

People love slogans. They love slogans because they need something to hang on to. A stake in the ground, a beacon they can point their sinking ship to in times of crisis.

The latest such beacon is AGILE.

I must admit, having been implementing software projects in both Agile and Waterfall environments, I’m not a fanatic, one way or another. Horses for courses. Nothing in life is absolute so there is absolutely no reason to assume that one model or another will always deliver better results – it all depends on the circumstances, more specifically the readiness of the specific environment and organizational maturity to operate within the operational constraints imposed / required by that methodology or the other.

If you gauge the public opinion, as reflected in recent blog articles, you would conclude that the wind is blowing Agile. Experienced project managers are progressively casting their attention at this model and the message they are sending to their devoted readership is “agile is good” and “agile can solve your ailing problems” – so much so that the PMI has now decided to jump on the publicity wagon and institute yet another certification – yes, you guessed it, Agile PM certification.

I wouldn’t be overly concerned about this fashionable trend if not for the fact that such public relation campaign does matter. As a recent article in the NewScientist (“Following the herd actually shifts your opinion“) shows, such publications carry a lot of weight as they are likely to result in shifting in public opinion for no other reason than that it gets publicity and air-time.

One thing I find really funny / annoying is the attempt by some writers to re-write their experience and mask it behind “we’ve always applied Agile in our projects, sure we didn’t quite call it Agile, but if you look at the processes we’ve used, they were just like what Agile prescribes”.

So here’s my stand on this question: Agile is good, but so are other software development methodologies, provided that you apply them appropriately and provided that you manage them correctly. One way or another you’ve got to do things ‘right’. If you rely on the methodology to save your day, think again. It is not the methodology that brings the results but the way people implement that methodology.

Think about it!

Introduction

The purpose of this article is to illustrate the way in which the Earned Value Measurement (EVM) approach is introduced into an Agile development project. The reason I chose to write about this topic is because the majority of the literature dealing with Earned Value is focused on the implementation and utilization of Earned Value techniques in waterfall based implementations and not so much on the adaptability of this technique to other types of software development implementations, like Agile, Critical Chain etc.

Major concepts in EVM

The EVM is being heavily promoted as a key set of tools for determining cost and schedule variances in projects, based on work performed against pre-established project baselines. The Project Management Institute (PMI) through its Project Management Body of Knowledge (PMBOK©) is encouraging project managers to use Earned Value as a key Tool and Technique in a number of Knowledge Areas, and Earned Value related Outputs are used as key Performance Measurements used to validate the project’s performance against its plan.

At the heart of the EVM lies the concept that as the project progresses, value is being generated. This value, measured in $’s, normally called “Earned Value” or EV, can then be compared to the project’s actual costs (AC) and planned value (PV) and this comparison process can assist in forecasting future project performance.

Key parameters used as part of the EVM include:

Successful utilization of EVM is dependent on early determination of project baselines (such that BAC and PV are easily identified) as well as on the ability of the organization to record and retrieve actual progress made against the project and being able to report on actual costs incurred to make this progress happen.

Major concepts in Agile

The Agile development approach is in fact a collective name for a variety of agile based methodologies that got united under the banner of the Agile Manifesto. The core principle behind the agile movement was the realization that better, faster, more responsive and cheaper development can be achieved via the introduction and adoption of a number of principles, including iterative development, constant customer collaboration and faster adaptation to scope changes.

For the purpose of our discussion I would like to define the following terms:

Incorporating Agile and EVM

As will be shown in the example below, applying EVM concepts in an Agile project requires three basic pieces of information:

1. The Product Backlog in points – i.e. what is the total scope of development for this project, presented as a number of points.
2. Baseline Velocity – i.e. a planned  value of the total number of points planned to be delivered / complete during each Iteration.
3. Cost Per Point – An estimated cost for delivering a single Point. This would normally be based on past performance of the delivering organization.

Using the above parameters, two further planning measurements can be derived:

1. The Budgeted At Completion (BAC) – this is the product of multiplying the Product Backlog by the Cost Per Point. So, for example, if the total scope of work for the project is estimated at 1,000 points, and the historical Cost Per Point is $1,000, then the total estimated cost of the project is 1,000 x $1,000 = $1M.
2. Planned number of Iterations – this can be derived from dividing the Product Backlog by the Baseline Velocity. For example, if the total scope of work for the project is estimated at 1,000 points, and the Baseline Velocity is 40 points per Iteration, then the planned number of Iterations is 1,000 / 40 = 25 Iterations.

Traditionally most Agile organizations have a clear view of their Baseline Velocity. This performance measurement is an indicator that reflects their on-going and historical efficiency and as such it is a measure that is being kept up-to-date and gets monitored on a regular basis.

More problematic is the up-front quantification of the project’s scope. Determining total project scope does not go against the underlying principles of the Agile Software Development Methodology, but is often seen by hardcore ‘Agile’ists as contravening with one of Agile’s tenants which encourages flexible change management and constant adaptation to customer’s changing needs. Notwithstanding the above, some level of total scope quantification is required in order to integrate EVM measurements into the project delivery cycle. This can be done upfront, or, if not possible, can be done using the Rolling Wave Planning approach (a tool recognized by the PMBOK) where progressive elaboration over a period of time is used to further expand the level and detail of project planning.

Determining the Cost per Point is another area requiring attention before implementing EVM in an Agile development environment. Having access to this information requires the organization to track it’s development costs such that they include not just the direct development costs (i.e. direct development resources) but also the cost associated with all other supporting disciplines (e.g. Business Analysts, Testers, Project Managers, etc.). That, obviously, would be the ideal scenario but even if that wasn’t possible, because of internal organizational deficiencies, other options could also be used. For example, knowing only the direct development costs could be used to extrapolated the total costs, assuming that development consists of x% of the total project costs. But, to achieve best possible cost estimates it would be beneficial for the organization to utilize timesheet functionality such that past costs can be analyzed and a more realistic and accurate cost estimates can be derived.

Some explanation is required to clarify the method used to determine the Earned Value produced by the end of each Iteration. As detailed above, Earned Value is the value attached to the Project’s throughput. Since we are using Points to describe the level of complexity associated with the delivered functionality and since we are using the combination of total points (i.e. the Backlog) and the Cost Per Point to derive the total budgetary requirements for the project, we can safely say that the value of each delivered point is the proportion of that point off the total cost of the project. So, for instance, in a project with a total backlog of 1,000 points and a budget of $1.5m, a completion of 100 points (i.e. 10% of the total scope) will deliver value of 10% * $1.5m = $150k (i.e. EV = $150,000).

Agile / EVM – example

The following example is based on a real project but has been condensed and numbers simplified in order to illustrate the way EVM can be used to measure project performance in an Agile development environment:

As discussed above, the only inputs necessary to enable EVM performance monitoring for the project are the Product Backlog, the planned Velocity and the projected Cost per Point (all three are highlighted with a yellow background in the table above). In our example:

1. the Backlog was  estimated at being 200 points,
2. the Velocity is estimated at 25 points per Iteration and
3. the Cost Per Point was estimated at $1,600.

Given the above parameters the BAC and # of Iterations were calculated as:

1. BAC = $1,600 x 200 = $320,000.
2. # of Iterations = 200 / 25 = 8

Also pre-calculated are the following parameters:

1. Planned % Complete per Iteration = 200 / 25 = 12.5%
2. Planned Value per Iteration = 12.5% x $320,000 = $40,000

The Iteration is marked as Iteration 0 as the planning stage for the project is done before actual development work commences.

With the above baseline established for the project and a project start date of Jan 5th 2009, the first Iteration has recorded the following results:

Notice that the monitoring spreadsheet is programmed to calculate the baseline project end date. Given a commencement date of Monday, Jan 5th and an expected duration of 8 iterations (each with duration of 2 calendar weeks) it is expected that the project will be complete by Fri, May 8th.

The two parameters without which EVM calculations cannot proceed are the Actual Cost for the Iteration (AC) and the actual number of points complete. In our case AC = $30,000 and the number points complete = 20.

Given the above we are now in a position to fully exploit all other EVM parameters, as follows:

1. The number of points actually complete during this iteration equate to 10% of the total backlog (20 out of 200). We can now use this figure to derive the Iteration’s Earned Value being 10% of the total project’s value, or $320,000 x 10% = $32,000.
2. The remaining parameters are easily calculated as per the formulas on the left.
3. The CPI is > 1 which implies that the value of the project’s throughput is higher than the cost of production.
4. The SPI is < 1 which implies that the value of the project’s throughput is below the planned value.
5. With current performance we’ll be under budget but will not be meeting the project’s scheduled completion date.

Estimated Total # of iterations at Completion is derived from dividing the total number of planned iterations (8) by the SPI (0.80). Planned completion date at this point, based on current performance, is Jun 5th.

As the 2nd Iteration was progressing is has become apparent that some key functionality has been incorrectly scoped. This has led to a re-estimation of the total project’s backlog and has resulted in the following:

The total scope (represented by the baseline backlog) has increased from 200 points to 250 points. This affected two of our basic baseline parameters:

1. The total number of planned iterations has increased from 8 to 10, and
2. The budget baseline has increased from $320,000 to $400,000.

No further planning adjustments have been made (i.e. Velocity and Cost Per Point have not been modified).

The 3rd Iteration, commencing Feb 2nd has taken into account the changes in the project’s scope and the actual results for the 3rd and 4th iterations were as follows:

During the 3rd and 4th Iteration the team’s velocity increased by a factor of 50% (from 20 to 30 points per iteration). The increased throughput resulted in the following:

1. Increasingly positive Cost Variance.
2. Improvement in the Schedule Variance to the point where at the end of the 4th Iteration it equaled 0.
3. With the above performance we could derive that all things being equal, the project will deliver on time and under budget.

Summary and Conclusions:

The above example represents a basic implementation of EVM and as such does not utilize some of the more advanced and esoteric variables available within the EVM world. Notwithstanding the above, the discussion above demonstrates how this simple approach can deliver an effective performance measurement solution in an Agile development environment without impacting the team’s velocity. Adding cost measurements to the traditional Agile burn rate can result in better acceptance of the Agile development philosophy in development environments that would traditionally shy away from such an approach due to it’s perceived lack of rigor and formality. Earned Value measurements are excellent communication indicators which can be shared with all project stakeholders including the development team, so they gain a better understanding and insight into the financial impact of their performance. Last but not least, for the Project Manager, responsible for the financial performance of the project, utilizing EVM as part of his or her project management tools can ensure that better pro-active actions can be taken to stir the project in the right direction.

If you haven’t implemented EVM and have a burning desire to use this technique in your next Agile project I suggest you follow the following steps:

1. Start Small – Use a small to medium size project (2-4 months long) to trial out your data collection and performance measurement interpretation techniques.
2. Make sure you have the necessary pre-requisites in place and if not, take concrete steps to have them established. For instance, if your organization does not have a time recording (timesheet) system in place, introduce a simple spreadsheet in which daily effort data, per team member, can be recorded. But be careful – don’t complicate the process and don’t attempt to record data at unrealistic or impractical levels of granularity as this will put your team members off and your chances at collecting reliable and accurate data will rapidly diminish.
3. Remember, you only need three basic planning parameters identified to begin with: Backlog, Velocity and Cost. These will provide you with the baseline measures for your scope, throughput and cost.
4. In order to calculate your actual costs you will might need to also collect the real cost per day for each of your resources. Try your HR or Finance departments, as they more than likely keep track of this data. With this data and the actual data collected in your timesheet system you will have no difficulty in calculating your actual costs.
5. Give it a go – it is not difficult and both you and your stakeholders will derive much benefit from it.

As usual, hope you enjoyed this post. If you did, why not share this with others by using the ‘Sharing is caring’ buttons below. If you want to read this and future posts using your news reader use the ‘Subscribe’ option at the top right corner of this page.

Until next time.