Difference between revisions of "Server Room Shutdown Procedure"

From CSLabsWiki
m (Expected Outage)
m (Expected Outage)
Line 30: Line 30:
 
</script>
 
</script>
 
<ul style="list-style:none; margin-left:3em;">
 
<ul style="list-style:none; margin-left:3em;">
<li><span onclick="toggle(this)"><span>□</span> Shutdown Blade Center</span>
+
<li><span onclick="toggle(this)"><span>□</span> Shutdown Animal</span>
 
<ul style="list-style:none; margin-left:3em;">
 
<ul style="list-style:none; margin-left:3em;">
<li><span onclick="toggle(this)"><span>□</span> All Blades</span>
+
<li><span onclick="toggle(this)"><span>□</span> Shutdown System</span>
 
<ul style="list-style:none; margin-left:3em;">
 
<ul style="list-style:none; margin-left:3em;">
<li><span onclick="toggle(this)"><span>□</span> If possible, use Ctrl-Alt-Delete to initialte a shutdown.</span></li>
+
<li><span onclick="toggle(this)"><span>□</span> shutdown -h now</span></li>
<li><span onclick="toggle(this)"><span>□</span> If previous does not work, press the power button on the front of the blade</span></li>
 
 
</ul>
 
</ul>
 
</li>
 
</li>
 
</ul>
 
</ul>
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown Mirror</span>
</li>
 
<li><span onclick="toggle(this)"><span>□</span> Shutdown Animal</span>
 
 
<ul style="list-style:none; margin-left:3em;">
 
<ul style="list-style:none; margin-left:3em;">
<li><span onclick="toggle(this)"><span>□</span> Killall "vbladed" exports</span></li>
+
<li><span onclick="toggle(this)"><span>□</span> Shutdown System</span>
  +
<ul style="list-style:none; margin-left:3em;">
<li><span onclick="toggle(this)"><span>□</span> Shutdown Machine</span></li>
 
  +
<li><span onclick="toggle(this)"><span>□</span> shutdown -h now</span></li>
  +
</ul>
  +
</li>
  +
</ul>
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown Xen1</span>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown All VMs</span>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> xm shutdown -a</span></li>
  +
</ul>
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown System</span>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> shutdown -h now</span></li>
  +
</ul>
  +
</li>
  +
</ul>
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown Xen2</span>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown All VMs</span>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> xm shutdown -a</span></li>
  +
</ul>
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown System</span>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> shutdown -h now</span></li>
  +
</ul>
  +
</li>
 
</ul>
 
</ul>
</li>
 
 
<li><span onclick="toggle(this)"><span>□</span> Shutdown Righteous</span>
 
<li><span onclick="toggle(this)"><span>□</span> Shutdown Righteous</span>
 
<ul style="list-style:none; margin-left:3em;">
 
<ul style="list-style:none; margin-left:3em;">
<li><span onclick="toggle(this)"><span>□</span> Shutdown Machine</span></li>
+
<li><span onclick="toggle(this)"><span>□</span> Shutdown All VMs</span>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> xm shutdown -a</span></li>
  +
</ul>
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown System</span>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> shutdown -h now</span></li>
  +
</ul>
  +
</li>
 
</ul>
 
</ul>
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown Blade Center</span>
</li>
 
<li><span onclick="toggle(this)"><span>□</span> Shutdown Windows Server</span>
 
 
<ul style="list-style:none; margin-left:3em;">
 
<ul style="list-style:none; margin-left:3em;">
<li><span onclick="toggle(this)"><span>□</span> ITL KVM Port 4, 3, 2, 1 - Select "Shutdown" on Login Screen</span></li>
+
<li><span onclick="toggle(this)"><span>□</span> All Blades</span>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> If possible, use Ctrl-Alt-Delete to initialte a shutdown.</span></li>
  +
<li><span onclick="toggle(this)"><span>□</span> If previous does not work, press the power button on the front of the blade.</span></li>
  +
</ul>
  +
</li>
 
</ul>
 
</ul>
 
</li>
 
</li>
<li><span onclick="toggle(this)"><span>□</span> ITL Rack Power (on top of Cisco 7000s on bottom of each rack)</span>
+
<li><span onclick="toggle(this)"><span>□</span> ITL Rack</span>
 
<ul style="list-style:none; margin-left:3em;">
 
<ul style="list-style:none; margin-left:3em;">
<li><span onclick="toggle(this)"><span>□</span> Kill 4, 3, 2, 1</span></li>
+
<li><span onclick="toggle(this)"><span>□</span> Shutdown All Systems</span>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> If possible, use Ctrl-Alt-Delete to initialte a shutdown.</span></li>
  +
<li><span onclick="toggle(this)"><span>□</span> If previous does not work, press the power button on the front of the blade.</span></li>
  +
</ul>
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown Power Strips (on top of Cisco 7000s on bottom of each rack)</span>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> Kill 4, 3, 2, & 1. In that order.</span></li>
  +
</ul>
  +
</li>
 
</ul>
 
</ul>
</li>
 
 
<li><span onclick="toggle(this)"><span>□</span> Unisys Server</span>
 
<li><span onclick="toggle(this)"><span>□</span> Unisys Server</span>
 
<ul style="list-style:none; margin-left:3em;">
 
<ul style="list-style:none; margin-left:3em;">
Line 69: Line 113:
 
<li><span onclick="toggle(this)"><span>□</span> Brown Rack</span>
 
<li><span onclick="toggle(this)"><span>□</span> Brown Rack</span>
 
<ul style="list-style:none; margin-left:3em;">
 
<ul style="list-style:none; margin-left:3em;">
<li><span onclick="toggle(this)"><span>□</span> Must kill at circuit breaker</span></li>
+
<li><span onclick="toggle(this)"><span>□</span> Unplug from wall</span></li>
 
</ul>
 
</ul>
 
</li>
 
</li>

Revision as of 10:08, 21 August 2009


The following covers the steps necessary to shutdown the equipment in the server room in the event of an emergency or scheduled outage.

Unexpected Power Interruption

  • In the event of a sudden loss of power in the labs, a majority of equipment in the server room will lose power immediately.


  • Several key systems (Righteous, Xen1, Xen2, Mirror, & Animal) and the network will remain online anywhere from 3-20 minutes depending on the load on the particular UPS they are plugged into. These key systems will automatically shutdown prior to the full depletion of the backup power provided by the UPSs. It is best to let this process occur automatically since it will allow most of the systems to return to normal operation when power returns and system downtime will be minimal.


  • Upon the return of power, a number of systems may automatically power on depending on their particular settings. Most of the key systems (listed above) & the network will automatically power back on when power returns; however, Animal will have to be manually turned on and some of the other systems might as well depending on when they shutdown and when power returned to the labs. All VMs should automatically start up when their host system finishes booting.


Expected Outage

<html> <script type="text/javascript"> function toggle(obj){

 if(obj.firstChild.innerHTML!='x'){
   obj.firstChild.innerHTML='x';
   obj.style.textDecoration='line-through';
   obj.style.color='#999';
 }else{
   obj.firstChild.innerHTML='□'
   obj.style.textDecoration='none';
   obj.style.color='#000';
 }

} </script>

  • Shutdown Animal
    • Shutdown System
      • shutdown -h now
  • Shutdown Mirror
    • Shutdown System
      • shutdown -h now
  • Shutdown Xen1
    • Shutdown All VMs
      • xm shutdown -a
    • Shutdown System
      • shutdown -h now
  • Shutdown Xen2
    • Shutdown All VMs
      • xm shutdown -a
    • Shutdown System
      • shutdown -h now
  • Shutdown Righteous
    • Shutdown All VMs
      • xm shutdown -a
    • Shutdown System
      • shutdown -h now
  • Shutdown Blade Center
    • All Blades
      • If possible, use Ctrl-Alt-Delete to initialte a shutdown.
      • If previous does not work, press the power button on the front of the blade.
  • ITL Rack
    • Shutdown All Systems
      • If possible, use Ctrl-Alt-Delete to initialte a shutdown.
      • If previous does not work, press the power button on the front of the blade.
    • Shutdown Power Strips (on top of Cisco 7000s on bottom of each rack)
      • Kill 4, 3, 2, & 1. In that order.
  • Unisys Server
    • Shutdown Management Module (from login prompt)
    • Shutdown lower unit (from the front panel)
  • Brown Rack
    • Unplug from wall
  • Power at Circut Breaker
    • Blade Center - 6 Large Circuits (COSI side) (Double 1-6)
    • Plugs on top of COSI side breaker - 3 Small Circuits (COSI side) (Single 1-3)
    • ITL Side / Labs - See label on breaker door

</html>