Difference between revisions of "Server Room Shutdown Procedure"

From CSLabsWiki
m
m (fixed this... apparently I goofed somewhere)
Line 30: Line 30:
 
<li><span onclick="toggle(this)"><span>□</span> Shutdown Animal</span>
 
<li><span onclick="toggle(this)"><span>□</span> Shutdown Animal</span>
 
<ul style="list-style:none; margin-left:3em;">
 
<ul style="list-style:none; margin-left:3em;">
<li><span onclick="toggle(this)"><span>□</span> Shutdown System</span>
+
<li><span onclick="toggle(this)"><span>□</span> Shutdown Machine (/sbin/shutdown -h now)</span></li>
<ul style="list-style:none; margin-left:3em;">
 
<li><span onclick="toggle(this)"><span>□</span> shutdown -h now</span></li>
 
</ul>
 
</li>
 
 
</ul>
 
</ul>
  +
</li>
<ul style="list-style:none; margin-left:3em;">
 
 
<li><span onclick="toggle(this)"><span>□</span> Shutdown Storage</span>
 
<li><span onclick="toggle(this)"><span>□</span> Shutdown Storage</span>
 
<ul style="list-style:none; margin-left:3em;">
 
<ul style="list-style:none; margin-left:3em;">
<li><span onclick="toggle(this)"><span>□</span> Shutdown System</span>
+
<li><span onclick="toggle(this)"><span>□</span> Shutdown All VMs (/usr/sbin/xm )</span></li>
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown Machine (/sbin/shutdown -h now)</span></li>
<ul style="list-style:none; margin-left:3em;">
 
<li><span onclick="toggle(this)"><span>□</span> shutdown -h now</span></li>
 
</ul>
 
</li>
 
 
</ul>
 
</ul>
  +
</li>
 
<li><span onclick="toggle(this)"><span>□</span> Shutdown Mirror</span>
 
<li><span onclick="toggle(this)"><span>□</span> Shutdown Mirror</span>
 
<ul style="list-style:none; margin-left:3em;">
 
<ul style="list-style:none; margin-left:3em;">
<li><span onclick="toggle(this)"><span>□</span> Shutdown System</span>
+
<li><span onclick="toggle(this)"><span>□</span> Shutdown Machine (/sbin/shutdown -h now)</span></li>
<ul style="list-style:none; margin-left:3em;">
 
<li><span onclick="toggle(this)"><span>□</span> shutdown -h now</span></li>
 
</ul>
 
</li>
 
 
</ul>
 
</ul>
  +
</li>
 
<li><span onclick="toggle(this)"><span>□</span> Shutdown Xen1</span>
 
<li><span onclick="toggle(this)"><span>□</span> Shutdown Xen1</span>
 
<ul style="list-style:none; margin-left:3em;">
 
<ul style="list-style:none; margin-left:3em;">
<li><span onclick="toggle(this)"><span>□</span> Shutdown All VMs</span>
+
<li><span onclick="toggle(this)"><span>□</span> Shutdown All VMs (/usr/sbin/xm )</span></li>
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown Machine (/sbin/shutdown -h now)</span></li>
<ul style="list-style:none; margin-left:3em;">
 
<li><span onclick="toggle(this)"><span>□</span> xm shutdown -a</span></li>
 
</ul>
 
<li><span onclick="toggle(this)"><span>□</span> Shutdown System</span>
 
<ul style="list-style:none; margin-left:3em;">
 
<li><span onclick="toggle(this)"><span>□</span> shutdown -h now</span></li>
 
</ul>
 
</li>
 
 
</ul>
 
</ul>
  +
</li>
 
<li><span onclick="toggle(this)"><span>□</span> Shutdown Xen2</span>
 
<li><span onclick="toggle(this)"><span>□</span> Shutdown Xen2</span>
 
<ul style="list-style:none; margin-left:3em;">
 
<ul style="list-style:none; margin-left:3em;">
<li><span onclick="toggle(this)"><span>□</span> Shutdown All VMs</span>
+
<li><span onclick="toggle(this)"><span>□</span> Shutdown All VMs (/usr/sbin/xm )</span></li>
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown Machine (/sbin/shutdown -h now)</span></li>
<ul style="list-style:none; margin-left:3em;">
 
<li><span onclick="toggle(this)"><span>□</span> xm shutdown -a</span></li>
 
</ul>
 
<li><span onclick="toggle(this)"><span>□</span> Shutdown System</span>
 
<ul style="list-style:none; margin-left:3em;">
 
<li><span onclick="toggle(this)"><span>□</span> shutdown -h now</span></li>
 
</ul>
 
</li>
 
 
</ul>
 
</ul>
  +
</li>
 
<li><span onclick="toggle(this)"><span>□</span> Shutdown Righteous</span>
 
<li><span onclick="toggle(this)"><span>□</span> Shutdown Righteous</span>
 
<ul style="list-style:none; margin-left:3em;">
 
<ul style="list-style:none; margin-left:3em;">
<li><span onclick="toggle(this)"><span>□</span> Shutdown All VMs</span>
+
<li><span onclick="toggle(this)"><span>□</span> Shutdown All VMs (/usr/sbin/xm )</span></li>
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown Machine (/sbin/shutdown -h now)</span></li>
<ul style="list-style:none; margin-left:3em;">
 
<li><span onclick="toggle(this)"><span>□</span> xm shutdown -a</span></li>
 
</ul>
 
<li><span onclick="toggle(this)"><span>□</span> Shutdown System</span>
 
<ul style="list-style:none; margin-left:3em;">
 
<li><span onclick="toggle(this)"><span>□</span> shutdown -h now</span></li>
 
</ul>
 
</li>
 
 
</ul>
 
</ul>
  +
</li>
 
<li><span onclick="toggle(this)"><span>□</span> Shutdown Blade Center</span>
 
<li><span onclick="toggle(this)"><span>□</span> Shutdown Blade Center</span>
 
<ul style="list-style:none; margin-left:3em;">
 
<ul style="list-style:none; margin-left:3em;">

Revision as of 12:15, 17 September 2009

The following covers the steps necessary to shutdown the equipment in the server room in the event of an emergency or scheduled outage.

Unexpected Power Interruption

  • In the event of a sudden loss of power in the labs, a majority of equipment in the server room will lose power immediately.


  • Several key systems (Righteous, Xen1, Xen2, Mirror, & Animal) and the network will remain online anywhere from 3-20 minutes depending on the load on the particular UPS they are plugged into. These key systems will automatically shutdown prior to the full depletion of the backup power provided by the UPSs. It is best to let this process occur automatically since it will allow most of the systems to return to normal operation when power returns and system downtime will be minimal.


  • Upon the return of power, a number of systems may automatically power on depending on their particular settings. Most of the key systems (listed above) & the network will automatically power back on when power returns; however, Animal will have to be manually turned on and some of the other systems might as well depending on when they shutdown and when power returned to the labs. All VMs should automatically start up when their host system finishes booting.


Expected Outage

<html> <script type="text/javascript"> function toggle(obj){

 if(obj.firstChild.innerHTML!='x'){
   obj.firstChild.innerHTML='x';
   obj.style.textDecoration='line-through';
   obj.style.color='#999';
 }else{
   obj.firstChild.innerHTML='□'
   obj.style.textDecoration='none';
   obj.style.color='#000';
 }

} </script>

  • Shutdown Animal
    • Shutdown Machine (/sbin/shutdown -h now)
  • Shutdown Storage
    • Shutdown All VMs (/usr/sbin/xm )
    • Shutdown Machine (/sbin/shutdown -h now)
  • Shutdown Mirror
    • Shutdown Machine (/sbin/shutdown -h now)
  • Shutdown Xen1
    • Shutdown All VMs (/usr/sbin/xm )
    • Shutdown Machine (/sbin/shutdown -h now)
  • Shutdown Xen2
    • Shutdown All VMs (/usr/sbin/xm )
    • Shutdown Machine (/sbin/shutdown -h now)
  • Shutdown Righteous
    • Shutdown All VMs (/usr/sbin/xm )
    • Shutdown Machine (/sbin/shutdown -h now)
  • Shutdown Blade Center
    • All Blades
      • If possible, use Ctrl-Alt-Delete to initialte a shutdown.
      • If previous does not work, press the power button on the front of the blade.
  • ITL Rack
    • Shutdown All Systems
      • If possible, use Ctrl-Alt-Delete to initialte a shutdown.
      • If previous does not work, press the power button on the front of the blade.
    • Shutdown Power Strips (on top of Cisco 7000s on bottom of each rack)
      • Kill 4, 3, 2, & 1. In that order.
  • Unisys Server
    • Shutdown Management Module (from login prompt)
    • Shutdown lower unit (from the front panel)
  • Brown Rack
    • Unplug from wall
  • Power at Circut Breaker
    • Blade Center - 6 Large Circuits (COSI side) (Double 1-6)
    • Plugs on top of COSI side breaker - 3 Small Circuits (COSI side) (Single 1-3)
    • ITL Side / Labs - See label on breaker door

</html>