*NEW - The "Seven Sources Of Problems" Model.

[Dr. ITiL]

In order to prevent problems we need to first identify, understand and then take action around the actual sources of problems. This is different from eliminating the underlying root cause. Fortunately, there are only seven (one for each day of the week!)

Unfortuantely, there's are billions of combinations and flavours - but with focused effort we can set about tackling each of the Seven Sources using straight forward, simple methods, based on ITIL best practice.

The Seven Sources are:-

1. Acceptance into Production - it got into your production world, you let it in, it caused a problem

2. New Changes - you let it through CAB, they implemented it, it caused a problem

3. Upgrades / Patches - you let them upgrade it (with or without change control), it caused a problem

4. Vendors / Suppliers - All of these categories, but considering suppliers instead of your local environment, they caused you a problem!

5. User Error - You let your Customer's use a system that they could break (bit unfair - some users will try hard to break their systems - albeit unintentionally)

6. How Production is Executed - The way you run your systems (re-scheduling batch processes, clock changes, deleting a job and it's dependencies fall over, you know the kind of thing!)

7. "Failures" - Electricity failure, mechanical breakdowns, elastic bands snapping, you name it... it can fail!

Implemented carefully, these methods could dramatically reduce the frequency, severity and volume of incoming problems.


[Edited. Reason for edit - removal of promotional links]

[eisbergsk]

I have found that the high severity incidents are frequently the result of business growth - despite the best efforts of Availability & Capacity Planning, and all the testing at every level, sometimes a product is more successful than you imagined; sometimes a promotion really takes off. Has anyone else noticed this?

[wolfhard]

We have experienced an increase in Incidents by as much as 100% above regular levels during periods of software upgrades/patches.

Wolfhard

[eisbergsk]

Wolfhard, out of curiosity can you describe what reasons you discovered for the increase in incidents after software upgrades/patches and what infrastructure changes you would recommend to prevent or eliminate them from happening?
/Sharon

[wolfhard]

Hi Sharon,

the updates were downloaded via the internet to a large number of PCs at external customer sites across Europe. Most PCs had Win2000 or WinXP and normal users did not have admin rights to install this upgrade. Also normal users were not ready for a different looking GUI suddenly popping up on their screens and did not take the correct actions.

As PCs are located with external customers they are all configured differently (OS setup, installed programs, ..) so basically anything is possible.

When customers install the software they are obliged to also install ALL upgrades. This is why they were also not asked wether they wanted to download the upgrade but the upgrades are downloaded automatically. After restarting the software customers are promted to install the upgrade or quit the program. If no admin is available this leads to the user calling the HelpDesk.

Solution:
Upgrades were sent on CDROM with clear instructions for system administrators.

Kind regards
Wolfhard