Metrics vs. KPIs

[matrejekm]

Hi,

Last Monday I attended an ITIL conference prepared by some ITSM solution vendors. One presentation draw my attention.

A lady presented a reporting module of ticketing system. What I was wondering she used once "KPI" term and second "metric" term for the same elements. She also claimed that number of RfCs is a KPI...

Till this time I was considering KPI as a performance metric. What means it has to be one matric against some other value: e.g. number of RfCs against time period.

You cannot say that your KPI is a number of Incidents - it tells you nothing about your actual performance, but if you use it against user base - you'll have some information.

... or maybe you can?

What are KPIs, statistical metrics and metrics in general for you?
Do you distinguish them? Are you using metrics as number of RfCs as a KPI or you have it against some other value?

Regards,
Mariusz M.

[gramsay]

A metric is any standard of measurement - number of incidents logged, average time to log incident, percentage incidents resolved within agreed service level etc.

A Key Performance Indicator (KPI) is a metric that you have chosen that will give an indication of your performance and can be used as a driver for improvement. In general it's prefered to just chose a few KPIs (say 3 or 4) to focus on.

For instance the business may be looking at reducing the spare parts bill. A KPI for Service management might be spares cost per incident or number of spares returned unused - whatever is chosen to focus on a business aim. When the aim is achieved, the metric remains, so the spares cost can still be monitored but it is no longer a focus for improvement and a new KPI can be used to focus on improvement elsewhere.

Of course there are areas where the business driver is always the same - say reduce incident resolution time - so a KPI for incident management might be overall incident resolution time.

In my last job I found KPIs useful both as a long term strategic aim and as a short term driver for performance improvement in particular areas. They are often tied in to employee targets & bonus schemes.

As for number of incidents logged - this is certainly a measure of ITSM effectiveness. The better the service IT gives, the fewer incidents should occur.

[matrejekm]

Thanks,

I agree to all, but the last claim.

Number of Incidents says nothing about your performance if it is not compared to something.
OK you can consider number of Incident decreasing or increasing against 1 service is your effectiveness indicator.
But if you introduce a new service your total incident volume is increasing - does it mean you're not effective - no definitely not - you're just upsizing, but your reports will say - you are less effective.
That a false conclusion draw from this metric.

regards,
Mariusz M.

[gramsay]

Obviously many things can effect any metric. You gave one example - the introduction of a new service. This would have an effect on lots of metrics - answer rates, response times, etc - but it doesn't invalidate their use as metrics.

New conditions and assumptions must always be factored into reports. It can always be possible to disregard incidents logged against a new service for an agreed period of time - perhaps to the start of the year. Your incident level will still remain a valid metric and when you start including the new service in the report you'll have some data on which to base your expected incident level.

One of the primary aims of ITIL processes is to reduce the number of incidents. It must be measured and reported both against services and as a whole.

[raroa]

It is a really interesting dilemma that reducing incidents is indeed an objective but number of incidents is not a good metric to emasure that objective.

Incidents can go up with a new service, as above. they can also go up just because service desk service is improving: suddenly people bother to call when in the past they would not have. people dig out old issues they had long given up on getting help for.

A CRM can go talk to a business unit and stir up a flood of incidents.

An SLA might require them to log stuff

And so on... Paradoxically, and frustratingly, many of the "best practice" activities we perform and their results can have an increasing effect on incidents.

[dboylan]

I think that the difference between Metric and KPI is pretty much a matter of semantics and you can define them however you want. Having said that, when I taught the Practitioner courses, I used this example:

If you were to draw a graph with the values you were measuring on the Y axis and Time on the X axis, then a single point on that graph would be a Metric. In fact, any individual point on the graph is a Metric.

But if you then take multiple metrics and drew a line between them, you would have a Key Performance Indicator.

Mind you, the line is not the Key Performance Indicator, it is the Angle of line and its relationship to the Time axis that determines the Key Performance Indicator (e.g.; A 10% improvement in the month of Dec. A 20% decrease over last week).

I would then cover the fact that the reason you have Key Performance Indicators is that you are trying to reach a Critical Success Factor (CSF). Measuring the difference between points does very little if you haven't set a goal. The goal is the Critical Success Factor.

[raroa]

or to be realy pedantically semantic, a set of metric points is a performance indicator. Your key performance indicators are the much smaller set of those you have selected to track in order to decide if you are achieving goals or not. therefore they will be a hopefully small set of metrics that map to your CSFs or goals.

[jpgilles]

Hi all,

The number of incidents is really one metric I would consider interesting, however, when it comes to KPI, the definite one I'd love to have (which means I'd love to be able to calculate - which is not always easy/possible) is:

Total costs of incidents(*) for the company / week or month or year

which, for example can be calculated , for each incident as:
Formula 1 (optimal): exact cost of work hours lost + exact loss generated by unavailability
Formula 2 (estimates): duration x number of people affected x average hourly cost of people + average hourly loss due to incident


I think this is really a KPI than can be shared with and looked at by the business clients.


(*) incident being considered here as service disruption/degradation
_________________
JP Gilles

[jpgilles]

Coming back to the subject for additional input, I personnaly use the following definitions:

* metrics: can be any type of measurement , possibly with no specific meaning , like "number of incidents"
* indicator: a calculation based on metrics that provides indication or value, like average number of incidents per user per period (the above metric means nothing by itself: if your compagny bought another one , the number of incidents would significantly increase, whereas that indicator would tell you whether the integration was smooth or not)
* KPI: specific indicator delivering valuable information about your performance. Most of the time, the type of KPI I used is "percentage of achievement of a commitment in the SLA".

For example if your SLA says that Z % of the calls to the Help Desk are answered directly within 30 sec. You would have:

Metric 1 (M1): total number of calls (in a given period)
Metric 2 (M2) : number of calls answered wihin 30 sec (in the same period)
Indicator (I): % of calls answered within 30 sec (during that period); I=M2/M1
KPI (K) : ratio between the indicator and the SLA; K=X/Z

If we take numbers, say 90% for the objective (Z) and 84% for the real rate (I) , then the KPI is about 89% of achievement.

The main interest with this type of KPI (% of achievement of objectives) is that you can aggregate various indicators (response time, availability, capacity, delays, ...) into a single global one : how much dow we achieve our commitments (by using an average or a more complex function).
_________________
JP Gilles

[Fabien]

The metrics discussion is always an interesting one to me. One day, I've seen a presentation about an implementation in a large insurance company and half of the presentation was about all the great charts and metrics that they put in place. There were so many that I really lost focus. I could not remember whether it was successful for the business or not, and I don't even remember if anyone asked the question.

The point is this: a metric is just a measurement. A KPI is an indicator (a metric) that you have chosen, and agreed with your partners (whether internal to IT or with customers), that will determine whether you are meeting your critical success factors (CSF).

For instance, if you determine that one of your Critical Success Factors is "reducing the overall number of incidents", then the total number of incidents over a defined time would be a valid KPI for you. If you define your critical success factor as "Reducing the overall number of incidents to existing and controled services", then obviously you will want to be more granular.

Hope this helps...
_________________
BR,
Fabien Papleux

Accenture
Technology Consulting | Service Excellence
Red Badge Certified

Twitter @itilgeek

[CWL]

Dear All,

It was great to get everyone's view on the definition of KPI and Metric. I am working in a ITSC environment and we are partly following the ITIL disciplines. Currently we are looking for the list of metrics that we can measure for all the discipline listed in ITIL so that we can compare whether the current metrics that we are measuring as our KPI is sufficient. Would appreciate any information on where can I get this list of metrics based on industry best practices.

Thanks in advance.

[Yorkie]

Hi Mariusz M.

In some ways you need to work backwards.

What are you expecting to achieve from measuring?

A KPI must instigate an action to improve performance. Root cause analysis, using the KPI to quatify the changes in performance.

Best Practice starts with your Organisation. Sure, industry best practice can provide a benchmark, a framework, but what does your business demand in terms of customer service.

All too often we count widgets and wonder why after counting them we have no idea what to do with them.

Ask the question; how is this going to improve my delivery of service?

Creating an effective KPI is not that easy. KPIs ought to provide a proactive culture. Its pretty sad if we wait until our customers leave before we stop them leaving KPI= Number of cancelled accounts last month.

In wish you well on your quest.

Regards

[Diarmid]

Well. I learn something every day.

Perhaps I'll try Internet Explorer sometime
_________________
"Method goes far to prevent trouble in business: for it makes the task easy, hinders confusion, saves abundance of time, and instructs those that have business depending, both what to do and what to hope."
William Penn 1644-1718