How to handle Server Reboots?

[UKVIKING]

Well as usual mark takes what i write and makes sense of it.

hmm..

Basically if there are to be reboots or service restarts that are not part of the incident resolution action, then I want the action tracked and controlled
_________________
John Hardesty
ITSM Manager's Certificate (Red Badge)

Change Management is POWER & CONTROL. /....evil laughter

[Diarmid]

Why not simply have a reboot procedure that defines criteria, controls and authority? It would include what requires to be logged and who has to be informed and would refer back to the incident report that led to the action.

Reboot is an operational action requiring operational management.
It is not really a change, although it might encompass a change (e.g. of the server configuration) in which case it will have been requested through the management of that change.
It is not an incident (unless the server does it all by itself) although it will often be used to resolve an incident.

[asrilrm]

It's not that simple concerning roles and responsibiilties. It is a kind of "gray area" which I myself am still in debate with colleagues about where to put this under.
At last we tried to incorporate V3, "Service Request Management" to our V2 to take care of things like password reset, server reboots, etc.

It's not about how, it's more about who, or which process would take care of the reboot thing.

[Diarmid]

Yes; service request sounds right. And it is good to have that formal an approach because it helps to keep control. This can make a difference to how you write the "reboot procedure" because it might be that request management takes over some of the burden of control.

I was too explicit about roles. It depends on your organization structure. Think of operational management as a conceptual function.

Unfortunately I don't have access to version three (and version two is now from memory!).

[Tomahawk]

Don't agree that a reboot warrants an RFC. If the server is hung then service is not available. Incident Mgt will restore the service via a reboot therefore why is an RFC necessary. Nothing has been changed. If however the underlying cause has been identified and a permanent solution required involving a reboot and say a memory upgrade I would say an RFC is required.
Once worked at an organisation with very old infrastructure. Reboots were required almost daily across 1000 + servers to correct faults. Prior to my arrival RFCs were logged.... Not anymore.... Routine reboots are handled via Incident Mgt and recorded in the fault calls raised.

[asrilrm]

Tomahawk,

That's one thing that is still in debate until now.
I could only say that it depends on how you manage your CMDB.
If you include the status of the server (on/off, etc) in your CI, then rebooting involves a change, otherwise it might not be necessary to raise a RFC.
But even if you don't record the status, raising a RFC would be beneficial to ensure that the impact of the reboot could be managed. And the only mechanism for that purpose is using the change control.
(that would imply that Change Management is power and control, as I've seen in someone's signature bwahahaha)

Ouch, I think I made my jaw dislocated

Cheers,
Asril

[Ed]

[UKVIKING]

ARRGH

Folks, there is no hard true answer........

IT DEPENDS.....

The key point on rebooting servers w/ or w/o a RFC needs to center on whether the service that the server is providing will be impacted

------

If there is NO service being provided, then the service needs to be restored under IM actions (reboot server) and there is really no need for an RFC

----

If the service is running and being used and the incident resolution required a reboot of the server to clear the incident, then the RFC could be raised

----

It depends on the change, incident, & problem mgmt policies, the general policies around what engineers can do, the SLAs associated with services provided and the general atmosphere of the incident resolution teams or support teams

----HISTORICAL NOTE

FOR EXAMPLE: The Microsoft support team has had a tendancy to clear their incidents for microsoft o/s server or application based incidents by merely having the systems rebooted.... after all, power cycling the machine will usually get you back to square one

However, this can cause hardware issues etc as well as clear out any sort of trace of real issues within the applications or O/S. As well as causing issues for services that are dependant on these boxes.

After several reboot actions, there was a great hullabaloo about the loss of a service at a critical time because the engineer in question merely just rebooted the system and it took an hour for all the individuals using the applications to get re-connected.

A wave of the mgmt hand and ended up required that ALL system reboots require MGMT approval and through the change process

This cause problems as you can expect..

The reboot standard became a diktat rather than a well though out policy statement - because the MS team were actually being lazy as they were being chased to CLEAR ticket queues rather than SOLVE the incidents

S-----------------

History over

---------------

So if your org wants to have reboots controlled through RFCs make sure it is documented!!!!!!!!!!!!!!
_________________
John Hardesty
ITSM Manager's Certificate (Red Badge)

Change Management is POWER & CONTROL. /....evil laughter

[Mark-OLoughlin]

Hi,

I love the debate on this this subject.

It is about control, communications and approving the corrective actions.

In Ed's case yes if you have to keep rebooting old systems then declace windows of opportunity where you can do this (Once agreed with the business) - publish the times and log incidents or standard changes (either is ok for this tyoe of pre approved activity).

If the action to reboot the server is not because of the above - you need to get approval to go ahead an reboot is - remember one physical server can now run many apps / VM's so other can be affected in trying to fix the other issue (therefor control and approval is needed here). How you get control and approval as John said is not hard and fast written ruloes but ITIL loks for this under emergency change process. In reality this can be a process you make up if you want as long as you cover the key areas of control, communication and approval.
_________________
Mark O'Loughlin
ITSM / ITIL Consultant

[UrgentJensen]

The best way to do it is to reboot after everyone else has gone home, so they'll never know.

UJ
_________________
Did I just say that out loud?

(Beige badge)

[Ed]

[UKVIKING]

Ed,

No I dont. hate it that is

I just think it is going to be the undying thread

And .....

these are the servers that you are looking for
_________________
John Hardesty
ITSM Manager's Certificate (Red Badge)

Change Management is POWER & CONTROL. /....evil laughter

[Ed]

Thanks Mate

I wondered where the hell they had gone!!!!

Seriously, I think there is, in reality, no final answer to this one because we all have such entrenched views!

No not entrenched, firmly held and passionately believed in. And they are all valid!

Wow - what happened there, me being diplomatic (or trying to be)
_________________
Regards

Ed

[Sandys]

I really enjoyed this post.

I battled with this topic myelf when we went with Remedy 7, I lost the battle.

Personally, I say the server reboot is convered under incident if performed as an emergency resolution and is performed sortly after the incident is logged. If the reboot is to be scheduled for a quiet time (Service is just degraded or not critical) then it should be an RFC.

Because the "State" of the service is being changed, the higher powers decided all reboots are to be an RFC.

I learned long ago it's best to pick your battles and this one was not worth fighting because both methods work.
_________________
Sandy Spears
Change Manager

"Change Manager" - The person people love to hate!

[sparkz]

I've read through this rather hastily so I apologise if I've missed something, however...

Change management also addresses risk and impact so... What happens if this server does not come back up? Incident management has rebooted, the server has now failed completely, there was no risk analysis, no mitigation etc. etc.

Standard changes can certainly speed up this process, can also be reviewed on a regular basis to keep the risk and impact up to date, and keep an audit trail on successful or unsuccessful changes to CI's.

An incident would presumably remain open until a resolution or workaround was in place. But would not report on a failed reboot specifically.

I've also written this in haste so I hope it makes sense.

Mark.