quantmleap

Introduction

The purpose of this article is to illustrate the way in which the Earned Value Measurement (EVM) approach is introduced into an Agile development project. The reason I chose to write about this topic is because the majority of the literature dealing with Earned Value is focused on the implementation and utilization of Earned Value techniques in waterfall based implementations and not so much on the adaptability of this technique to other types of software development implementations, like Agile, Critical Chain etc.

Major concepts in EVM

The EVM is being heavily promoted as a key set of tools for determining cost and schedule variances in projects, based on work performed against pre-established project baselines. The Project Management Institute (PMI) through its Project Management Body of Knowledge (PMBOK©) is encouraging project managers to use Earned Value as a key Tool and Technique in a number of Knowledge Areas, and Earned Value related Outputs are used as key Performance Measurements used to validate the project’s performance against its plan.

At the heart of the EVM lies the concept that as the project progresses, value is being generated. This value, measured in $’s, normally called “Earned Value” or EV, can then be compared to the project’s actual costs (AC) and planned value (PV) and this comparison process can assist in forecasting future project performance.

Key parameters used as part of the EVM include:

Successful utilization of EVM is dependent on early determination of project baselines (such that BAC and PV are easily identified) as well as on the ability of the organization to record and retrieve actual progress made against the project and being able to report on actual costs incurred to make this progress happen.

Major concepts in Agile

The Agile development approach is in fact a collective name for a variety of agile based methodologies that got united under the banner of the Agile Manifesto. The core principle behind the agile movement was the realization that better, faster, more responsive and cheaper development can be achieved via the introduction and adoption of a number of principles, including iterative development, constant customer collaboration and faster adaptation to scope changes.

For the purpose of our discussion I would like to define the following terms:

Incorporating Agile and EVM

As will be shown in the example below, applying EVM concepts in an Agile project requires three basic pieces of information:

1. The Product Backlog in points – i.e. what is the total scope of development for this project, presented as a number of points.
2. Baseline Velocity – i.e. a planned  value of the total number of points planned to be delivered / complete during each Iteration.
3. Cost Per Point – An estimated cost for delivering a single Point. This would normally be based on past performance of the delivering organization.

Using the above parameters, two further planning measurements can be derived:

1. The Budgeted At Completion (BAC) – this is the product of multiplying the Product Backlog by the Cost Per Point. So, for example, if the total scope of work for the project is estimated at 1,000 points, and the historical Cost Per Point is $1,000, then the total estimated cost of the project is 1,000 x $1,000 = $1M.
2. Planned number of Iterations – this can be derived from dividing the Product Backlog by the Baseline Velocity. For example, if the total scope of work for the project is estimated at 1,000 points, and the Baseline Velocity is 40 points per Iteration, then the planned number of Iterations is 1,000 / 40 = 25 Iterations.

Traditionally most Agile organizations have a clear view of their Baseline Velocity. This performance measurement is an indicator that reflects their on-going and historical efficiency and as such it is a measure that is being kept up-to-date and gets monitored on a regular basis.

More problematic is the up-front quantification of the project’s scope. Determining total project scope does not go against the underlying principles of the Agile Software Development Methodology, but is often seen by hardcore ‘Agile’ists as contravening with one of Agile’s tenants which encourages flexible change management and constant adaptation to customer’s changing needs. Notwithstanding the above, some level of total scope quantification is required in order to integrate EVM measurements into the project delivery cycle. This can be done upfront, or, if not possible, can be done using the Rolling Wave Planning approach (a tool recognized by the PMBOK) where progressive elaboration over a period of time is used to further expand the level and detail of project planning.

Determining the Cost per Point is another area requiring attention before implementing EVM in an Agile development environment. Having access to this information requires the organization to track it’s development costs such that they include not just the direct development costs (i.e. direct development resources) but also the cost associated with all other supporting disciplines (e.g. Business Analysts, Testers, Project Managers, etc.). That, obviously, would be the ideal scenario but even if that wasn’t possible, because of internal organizational deficiencies, other options could also be used. For example, knowing only the direct development costs could be used to extrapolated the total costs, assuming that development consists of x% of the total project costs. But, to achieve best possible cost estimates it would be beneficial for the organization to utilize timesheet functionality such that past costs can be analyzed and a more realistic and accurate cost estimates can be derived.

Some explanation is required to clarify the method used to determine the Earned Value produced by the end of each Iteration. As detailed above, Earned Value is the value attached to the Project’s throughput. Since we are using Points to describe the level of complexity associated with the delivered functionality and since we are using the combination of total points (i.e. the Backlog) and the Cost Per Point to derive the total budgetary requirements for the project, we can safely say that the value of each delivered point is the proportion of that point off the total cost of the project. So, for instance, in a project with a total backlog of 1,000 points and a budget of $1.5m, a completion of 100 points (i.e. 10% of the total scope) will deliver value of 10% * $1.5m = $150k (i.e. EV = $150,000).

Agile / EVM – example

The following example is based on a real project but has been condensed and numbers simplified in order to illustrate the way EVM can be used to measure project performance in an Agile development environment:

As discussed above, the only inputs necessary to enable EVM performance monitoring for the project are the Product Backlog, the planned Velocity and the projected Cost per Point (all three are highlighted with a yellow background in the table above). In our example:

1. the Backlog was  estimated at being 200 points,
2. the Velocity is estimated at 25 points per Iteration and
3. the Cost Per Point was estimated at $1,600.

Given the above parameters the BAC and # of Iterations were calculated as:

1. BAC = $1,600 x 200 = $320,000.
2. # of Iterations = 200 / 25 = 8

Also pre-calculated are the following parameters:

1. Planned % Complete per Iteration = 200 / 25 = 12.5%
2. Planned Value per Iteration = 12.5% x $320,000 = $40,000

The Iteration is marked as Iteration 0 as the planning stage for the project is done before actual development work commences.

With the above baseline established for the project and a project start date of Jan 5th 2009, the first Iteration has recorded the following results:

Notice that the monitoring spreadsheet is programmed to calculate the baseline project end date. Given a commencement date of Monday, Jan 5th and an expected duration of 8 iterations (each with duration of 2 calendar weeks) it is expected that the project will be complete by Fri, May 8th.

The two parameters without which EVM calculations cannot proceed are the Actual Cost for the Iteration (AC) and the actual number of points complete. In our case AC = $30,000 and the number points complete = 20.

Given the above we are now in a position to fully exploit all other EVM parameters, as follows:

1. The number of points actually complete during this iteration equate to 10% of the total backlog (20 out of 200). We can now use this figure to derive the Iteration’s Earned Value being 10% of the total project’s value, or $320,000 x 10% = $32,000.
2. The remaining parameters are easily calculated as per the formulas on the left.
3. The CPI is > 1 which implies that the value of the project’s throughput is higher than the cost of production.
4. The SPI is < 1 which implies that the value of the project’s throughput is below the planned value.
5. With current performance we’ll be under budget but will not be meeting the project’s scheduled completion date.

Estimated Total # of iterations at Completion is derived from dividing the total number of planned iterations (8) by the SPI (0.80). Planned completion date at this point, based on current performance, is Jun 5th.

As the 2nd Iteration was progressing is has become apparent that some key functionality has been incorrectly scoped. This has led to a re-estimation of the total project’s backlog and has resulted in the following:

The total scope (represented by the baseline backlog) has increased from 200 points to 250 points. This affected two of our basic baseline parameters:

1. The total number of planned iterations has increased from 8 to 10, and
2. The budget baseline has increased from $320,000 to $400,000.

No further planning adjustments have been made (i.e. Velocity and Cost Per Point have not been modified).

The 3rd Iteration, commencing Feb 2nd has taken into account the changes in the project’s scope and the actual results for the 3rd and 4th iterations were as follows:

During the 3rd and 4th Iteration the team’s velocity increased by a factor of 50% (from 20 to 30 points per iteration). The increased throughput resulted in the following:

1. Increasingly positive Cost Variance.
2. Improvement in the Schedule Variance to the point where at the end of the 4th Iteration it equaled 0.
3. With the above performance we could derive that all things being equal, the project will deliver on time and under budget.

Summary and Conclusions:

The above example represents a basic implementation of EVM and as such does not utilize some of the more advanced and esoteric variables available within the EVM world. Notwithstanding the above, the discussion above demonstrates how this simple approach can deliver an effective performance measurement solution in an Agile development environment without impacting the team’s velocity. Adding cost measurements to the traditional Agile burn rate can result in better acceptance of the Agile development philosophy in development environments that would traditionally shy away from such an approach due to it’s perceived lack of rigor and formality. Earned Value measurements are excellent communication indicators which can be shared with all project stakeholders including the development team, so they gain a better understanding and insight into the financial impact of their performance. Last but not least, for the Project Manager, responsible for the financial performance of the project, utilizing EVM as part of his or her project management tools can ensure that better pro-active actions can be taken to stir the project in the right direction.

If you haven’t implemented EVM and have a burning desire to use this technique in your next Agile project I suggest you follow the following steps:

1. Start Small – Use a small to medium size project (2-4 months long) to trial out your data collection and performance measurement interpretation techniques.
2. Make sure you have the necessary pre-requisites in place and if not, take concrete steps to have them established. For instance, if your organization does not have a time recording (timesheet) system in place, introduce a simple spreadsheet in which daily effort data, per team member, can be recorded. But be careful – don’t complicate the process and don’t attempt to record data at unrealistic or impractical levels of granularity as this will put your team members off and your chances at collecting reliable and accurate data will rapidly diminish.
3. Remember, you only need three basic planning parameters identified to begin with: Backlog, Velocity and Cost. These will provide you with the baseline measures for your scope, throughput and cost.
4. In order to calculate your actual costs you will might need to also collect the real cost per day for each of your resources. Try your HR or Finance departments, as they more than likely keep track of this data. With this data and the actual data collected in your timesheet system you will have no difficulty in calculating your actual costs.
5. Give it a go – it is not difficult and both you and your stakeholders will derive much benefit from it.

As usual, hope you enjoyed this post. If you did, why not share this with others by using the ‘Sharing is caring’ buttons below. If you want to read this and future posts using your news reader use the ‘Subscribe’ option at the top right corner of this page.

Until next time.