Difference between revisions of "Server Room Shutdown Procedure"

From CSLabsWiki
m
Line 15: Line 15:
 
==Expected Outage==
 
==Expected Outage==
   
  +
<html>
 
  +
<script type="text/javascript">
 
  +
function toggle(obj){
  +
if(obj.firstChild.innerHTML!='x'){
  +
obj.firstChild.innerHTML='x';
  +
obj.style.textDecoration='line-through';
  +
obj.style.color='#999';
  +
}else{
  +
obj.firstChild.innerHTML='□'
  +
obj.style.textDecoration='none';
  +
obj.style.color='#000';
  +
}
  +
}
  +
</script>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown Blade Center</span>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> Hydrogen</span>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown All VMs</span></li>
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown Blade</span></li>
  +
</ul>
  +
</li>
  +
<li><span onclick="toggle(this)"><span>□</span> Helium</span>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown All VMs</span></li>
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown Blade</span></li>
  +
</ul>
  +
</li>
  +
<li><span onclick="toggle(this)"><span>□</span> Lithium</span>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown All VMs</span></li>
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown Blade</span></li>
  +
</ul>
  +
</li>
  +
<li><span onclick="toggle(this)"><span>□</span> 4,7-8,11-14</span>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> If possible, use Ctrl-Alt-Delete to initialte a shutdown.</span></li>
  +
<li><span onclick="toggle(this)"><span>□</span> Press Power Button on Front of Blade</span></li>
  +
</ul>
  +
</li>
  +
</ul>
  +
</li>
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown Animal</span>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> Killall "vbladed" exports</span></li>
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown Machine</span></li>
  +
</ul>
  +
</li>
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown Righteous</span>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown Machine</span></li>
  +
</ul>
  +
</li>
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown Windows Server</span>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> ITL KVM Port 4, 3, 2, 1 - Select "Shutdown" on Login Screen</span></li>
  +
</ul>
  +
</li>
  +
<li><span onclick="toggle(this)"><span>□</span> ITL Rack Power (on top of Cisco 7000s on bottom of each rack)</span>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> Kill 4, 3, 2, 1</span></li>
  +
</ul>
  +
</li>
  +
<li><span onclick="toggle(this)"><span>□</span> Unisys Server</span>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown Management Module (from login prompt)</span></li>
  +
<li><span onclick="toggle(this)"><span>□</span> Shutdown lower unit (from the front panel)</span></li>
  +
</ul>
  +
</li>
  +
<li><span onclick="toggle(this)"><span>□</span> Brown Rack</span>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> Must kill at circuit breaker</span></li>
  +
</ul>
  +
</li>
  +
<li><span onclick="toggle(this)"><span>□</span> Power at Circut Breaker</span>
  +
<ul style="list-style:none; margin-left:3em;">
  +
<li><span onclick="toggle(this)"><span>□</span> Blade Center - 6 Large Circuits (COSI side) (Double 1-6)</span></li>
  +
<li><span onclick="toggle(this)"><span>□</span> Plugs on top of COSI side breaker - 3 Small Circuits (COSI side) (Single 1-3)</span></li>
  +
<li><span onclick="toggle(this)"><span>□</span> ITL Side / Labs - See label on breaker door</span></li>
  +
</ul>
  +
</li>
  +
</ul>
  +
</html>
   
 
[[Category:Internal How-Tos]]
 
[[Category:Internal How-Tos]]

Revision as of 09:50, 21 August 2009


The following covers the steps necessary to shutdown the equipment in the server room in the event of an emergency or scheduled outage.

Unexpected Power Interruption

  • In the event of a sudden loss of power in the labs, a majority of equipment in the server room will lose power immediately.


  • Several key systems (Righteous, Xen1, Xen2, Mirror, & Animal) and the network will remain online anywhere from 3-20 minutes depending on the load on the particular UPS they are plugged into. These key systems will automatically shutdown prior to the full depletion of the backup power provided by the UPSs. It is best to let this process occur automatically since it will allow most of the systems to return to normal operation when power returns and system downtime will be minimal.


  • Upon the return of power, a number of systems may automatically power on depending on their particular settings. Most of the key systems (listed above) & the network will automatically power back on when power returns; however, Animal will have to be manually turned on and some of the other systems might as well depending on when they shutdown and when power returned to the labs. All VMs should automatically start up when their host system finishes booting.


Expected Outage

<html> <script type="text/javascript"> function toggle(obj){

 if(obj.firstChild.innerHTML!='x'){
   obj.firstChild.innerHTML='x';
   obj.style.textDecoration='line-through';
   obj.style.color='#999';
 }else{
   obj.firstChild.innerHTML='□'
   obj.style.textDecoration='none';
   obj.style.color='#000';
 }

} </script>

  • Shutdown Blade Center
    • Hydrogen
      • Shutdown All VMs
      • Shutdown Blade
    • Helium
      • Shutdown All VMs
      • Shutdown Blade
    • Lithium
      • Shutdown All VMs
      • Shutdown Blade
    • 4,7-8,11-14
      • If possible, use Ctrl-Alt-Delete to initialte a shutdown.
      • Press Power Button on Front of Blade
  • Shutdown Animal
    • Killall "vbladed" exports
    • Shutdown Machine
  • Shutdown Righteous
    • Shutdown Machine
  • Shutdown Windows Server
    • ITL KVM Port 4, 3, 2, 1 - Select "Shutdown" on Login Screen
  • ITL Rack Power (on top of Cisco 7000s on bottom of each rack)
    • Kill 4, 3, 2, 1
  • Unisys Server
    • Shutdown Management Module (from login prompt)
    • Shutdown lower unit (from the front panel)
  • Brown Rack
    • Must kill at circuit breaker
  • Power at Circut Breaker
    • Blade Center - 6 Large Circuits (COSI side) (Double 1-6)
    • Plugs on top of COSI side breaker - 3 Small Circuits (COSI side) (Single 1-3)
    • ITL Side / Labs - See label on breaker door

</html>