The PM process needs tools (KPI:s) to evaluate itself, other processes (IM, CM, etc) and to be evaluated by other processes (or management).
Your KPI:s should therefore (IMHO) be sorted into 3 groups, mainly:
1) KPIs for evaluating other processes
2) KPIs for internal evaluation
3) KPIs for external evaluation
Some examples (out of our shop and off the top of my head):
# incidents/MI that were submitted from IM to PM with FULL and adequate documentation
# incidents correctly matched against the KEDB
Add a few more, to evaluate the communication and routines between IM and PM as well as the IM process output in general
Do the same with CM, SLM et al. One or two well chosen KPIs will point to weaknesses!
2) Here you can use the majority of the "standard" PM KPIs, but try to concentrate on measuring effectiveness (length of problem queue, time needed to resolve problem, number of submitted Known Errors to the KEDB and on and on. Many of these can be used by others to scrutinize the PM process, see 3)
3) Use KPIs wisely here. Be prepared to be watched by others, and offer a short monthly report to mgmt and the surrounding processes - to begin with.
All in all - not all KPIs make sense in all workplaces. Stay sane, don´t overdo it. Keep the helicopter view, don´t become too granular. You will eventually find the (few) crucial ones that really MEAN something and they will help you improve things.
Hope you get the idea. Lots more to read in the excellent book "Metrics for IT Service Management" by ITSMF.
Regards /Richard _________________ ---------------- bragging line -----------------
V2 Combined Practitioner (service support)
Joined: Mar 04, 2008 Posts: 1890 Location: Helensburgh
Posted: Wed Aug 25, 2010 8:22 pm Post subject:
I'm not convinced that Problem Management can be meaningfully subjected to KPIs in the sense of statistics except about money.
The number of problems in the queue depends on the number of problems that surface (that might be usable as a measure of the quality of other processes from which these problems emerge, but not of Problem Management).
Problems are largely dissimilar to each other. The time it takes to determine the underlying cause (some call this root cause) is dependent on the complexity of the system and the quality of the symptoms as data. The time taken to fully resolve a problem is dependent on the complexity, cost and scale of the solution, and can also be affected by priorities for resources over fairly long periods.
There are two areas that can be addressed:
1. Costs - you want to establish that the cost of Problem Management is covered by the projected savings from incidents not occurring as a consequence.
2. Efficiency and effectiveness - in Problem Management a lot of this is down to judgement about about process and practices and more subject to logical analysis than statistical. Audit will play a part, but the main thrust must be improvement initiatives. The best confirmation of improvement will be an increase in projected benefit over cost. Projected benefit is itself part judgement since the actual cost of incidents can only be determined by observing them and that is what Problem Management is trying to prevent.
If you train your technical analysts they should arrive at causes and solutions quicker, but it is not easy to prove since each problem is different.
If you improve the flow of information and co-operation to and from other sections then your solutions should be arrived at quicker and they may be better solutions. Again, each problem is different...
All the above is less true in very large and/or complex systems if these experience a lot of problems, but many IT services will not have much useful statistical data about Problem Management. _________________ "Method goes far to prevent trouble in business: for it makes the task easy, hinders confusion, saves abundance of time, and instructs those that have business depending, both what to do and what to hope."
William Penn 1644-1718
Posted: Tue Aug 31, 2010 5:12 am Post subject: Re: KPI confusion
# of incidents
reduction in the # of incidents
Which one is a KPI? Some say both whilst others say reduction... Some even say the reduction is a metric. Ugh!
All KPIs are metrics but not all metrics are KPIs. KPIs are Key Performance Indicators. In other words, KPIs should be able to tell something about a particular performance aspect, in this case of the Problem Mgmt process. KPIs typically have target values associated with them and are associated with an objective.
When identifying KPIs that are useful for you, start with the goal of your process. Problem Mgmt's goal - simply said - is to reduce/prevent incidents. Typical objectives are to identify problems, to develop workarounds that can be use in response to incidents pending a permanent solution, to determine root causes, and to eliminate these root causes (known errors) from the environment. For each of these objectives you can probably define some KPIs that help you evaluate effectiveness and efficiency. Some examples:
- % of open problems with a documented workaround (you may want this % to be more than a certain target value)
- the % of problems for which a root cause was found within defined target times (assuming that you have/want such targets)
- number of recurring incidents that will be prevented going forward as a result of eliminating known errors
Diarmid posted some valid warnings that I agree with. Problems can vary so widely in complexity and in the time and effort it takes to complete root cause analysis and corrective action, that just looking at the numbers may make you jump to the wrong conclusions. You may find that in month Y the average root cause analysis took twice as long as in month X. Not necessarily because people did a lousy job or were focused on other things than Problem Mgmt, but just because the issues they worked on were much more complex in month Y. So keep that in mind.
The other thing to consider when looking at incident volume trends to see how well Problem Mgmt has been doing, is the fact that there are other factors at play. If Change Mgmt continues to approve tons of high-risk barely tested changes then it may completely negate Problem Mgmt's achievements. That is why I would rather look at the incident volume that was eliminated as a result of doing Problem Mgmt rather than at the overall incident volume. _________________ Manager of Problem Management
Fortune 100 Company
Posted: Tue Sep 21, 2010 12:15 am Post subject: Re: re
I would suggest few things on KPI monthly . You can compare with previous months or years
Total Volume of the tickets
Total Closed tickets
Mean to restore ( To check how far we are doing the best )
how many tickets occurred by a change ( problem due to change
I would argue that the first 2 are not KPIs at all, the 3rd could be a KPI for Incident Mgmt, and the last one - if tweaked a little bit - could be a KPI for Change Mgmt (i.e. the % of changes that introduced problems).
As I stated in my previous post: All KPIs are metrics but not all metrics are KPIs. _________________ Manager of Problem Management
Fortune 100 Company
iradek..you are missing the target mostly, those are mostly IM. And you forget "User" in your category list..
Problem management needs metrics on input and output, as well as internal. KPIs are those metrics that help you judge the effectiveness and health/maturity of the process. Not all metrics do that. KPI= KEY performance indicator..
# of known errors (and solutions) posted by PM in KEDB
will show how helpful the PM process is towards IM - but it needs combining with other KPIs in order to show the whole picture. (After a while, most errors ARE known, hence a reduction in the flow.)
You may want to read up on this, I believe. See the book tip above. _________________ ---------------- bragging line -----------------
V2 Combined Practitioner (service support)
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum