The challenge that was originally posed and is still relevant was to develop a key performance indicator, a single figure, for a mixed portfolio of change, that could go on an executive dashboard and mean something!
After a little thought I created just such a KPI and ran it successfully for over three years.
The organisation already had the foundations for a KPI, but struggled to distil the essence into a number that, if monitored, would aid management's decision making.
The Foundations
The foundations were a project dashboard and an existing BRAG rating system.
The dashboard was a summarised report of the most important active change projects for the organisation. While most important often correlated with largest, this was not always the case. A regulatory "must do" project would make it to the dashboard, while the refurbishment of the in-house cafe and showers would not, rebranding would, but the replacement of printers and scanners may well not. The list was agreed with stakeholders up front and was adjusted over time as priorities changed and some projects were delivered and new (unplanned) ones arose.
This dashboard was no more than two sides of A4 and contained around 10-15 and no more than 20 projects or change initiatives. If the number was greater it diffused management attention and did not aid decision making in terms of resource allocation, etc.
Every project/change initiative was already RAG rated each month. The B in BRAG related to blue and was used to indicate "closed". A project was reported as Blue in the month it was formally closed, and then removed from future dashboards.
Broadly speaking the BRAG ratings meant:
B = blue signified complete/closed.
R = red meant the project was not likely to deliver as expected and there were issues that needed urgent attention
A = amber meant project delivery was at risk, but issues were being managed
G = green meant the project is on target to deliver as expected
These are a synthesis of the key considerations time/cost/quality as applied to each project and in a reasonably mature environment contain a lot of implicit information.
Creating A Number
The first question was to consider if all projects on the Dashboard were equal? I decided that this was not the case and needed some simple weighting to emphasise the critical items. In general I assigned a weight of 10 to dashboard projects, but used a lesser weighting of 3 where it felt more appropriate.
For the purposes of the KPI any project not on the dashboard was disregarded, or given a weight of zero.
The next thing was to the score BRAG. I used B=5, G=5, A=3 and R=-1.
Please note the negative score for Red. The idea being that a project in trouble normally distracts attention from other efforts, and thus has a wider, detrimental impact on the change portfolio.
So, for each dashboard project I could create a performance number that was (project weight x score) and I could sum those up each month. This provided a single number, ie sum of (project weight x score), but its absolute value was subject to the changing consitution of the project dashboard.
The way over this was to initially express actual score as a fraction of the total possible score that month ie if every project was on track or completed (Blue or Green).
This produced a new number that was
sum of (project weight x score)
sum of (project weight x 5)
The temptation was to express this as a percentage, but when one uses percentages psychology kicks in an readers assume that 100% is the target. This did not feel right in a world where we are looking at actively managing risk and trying to do more with less.
This said please bear with me as I use percentages for a little longer.
If every project was Amber, ie delivery had some risk, but issues were being managed then the indicator would come out at 60% ie 3/5. This felt like the lowest boundary of acceptable performance.
While we thought it unlikely that we would ever score 100% we also recognised that such a score would probably indicate that we were not pushing our change capability hard enough, over-staffed, under ambitious, too cautious, etc. I ran the numbers for the previous year and equated the number produced with how the world of change had felt at the time, as best we could recall.
We finally settled on 80% as the upper boundary of our target performance, agreeing that should we breach that we would want to look at the book of work / capacity with a view to stretching ourselves.
Of course you may choose other boundaries, depending on your risk appetite, but the point is that there is a target band and it is unlikely to be at 100%.
With this in mind we can create a number in a range. I would suggest that the range be 0-5, but you can choose one that suits.
Thus the monthly performance number is
sum of (project weight x score) x 5
sum of (project weight x 5)
with the target performance band between 3 and 4.
Another advantage is that "happy reporting" ie the eternally optimistic (or political?) reporting of projects at Green will tend to push the KPI into the upper band and thus attract scrutiny and discussion - no bad thing.
Using the Number
There is one final tweak and that is to not use the monthly figure, but rather a three month moving average. The reason being that the health of projects do fluctuate and can change quickly. Effective change management is able to properly address these fluctuations and take remedial/mitigating action. If, however, the situation is not addressed over a period, say three months, then we want the indicator to flag this.
So the final KPI is
3-month moving average of sum of (project weight x score) x 5
sum of (project weight x 5)
This was then graphed and reported on monthly with trends monitored.
Graphing the KPIA few tips in the graphing.
- Limit the scale so that space is not wasted. I set lower limit of the y-scale to 2 as if we ever got below that we would have more serious trouble than scoring a KPI!!
- Smooth the graphline - it looks better, guards against "knee jerk" reactions and fits better with the trend value of this indicator.
- Set the target performance band as green, the upper (too comfortable?) band as amber, ie worthy of attention and the lower band as red, ie needing attention.
Conclusion
Of course it contains a lot of detail within a single number, and real understanding of issues around the delivery performance of a change portfolio needs deeper examination, but as a single repeatable, credible number that can be used on a high-level KPI dashboard and provide useful insight, I have found no better!
Very sensible, thanks for sharing!
ReplyDelete