Difference between revisions of "Hydra"
(→Virtual Machine Core Allocation) |
|||
| Line 2: | Line 2: | ||
|ip_addr = 128.153.145.42 |
|ip_addr = 128.153.145.42 |
||
|contact_person = [[User:Jared|Jared Dunbar]] & [[User:bruskajp|James Bruska]] |
|contact_person = [[User:Jared|Jared Dunbar]] & [[User:bruskajp|James Bruska]] |
||
| − | |last_update = '' |
+ | |last_update = ''Spring 2017'' |
|services = VM Host |
|services = VM Host |
||
|category = Machines |
|category = Machines |
||
| Line 12: | Line 12: | ||
| maintainer = [[User:Jared|Jared Dunbar]] & [[User:bruskajp|James Bruska]] |
| maintainer = [[User:Jared|Jared Dunbar]] & [[User:bruskajp|James Bruska]] |
||
| hostname = hydra.cslabs.clarkson.edu |
| hostname = hydra.cslabs.clarkson.edu |
||
| − | | operating_system = |
+ | | operating_system = Arch Linux |
| interface1 = {{Network Interface | name=Clarkson Network | mac=0c:c4:7a:c6:e7:88 | ip=128.153.145.42}} |
| interface1 = {{Network Interface | name=Clarkson Network | mac=0c:c4:7a:c6:e7:88 | ip=128.153.145.42}} |
||
| cpuspecs = 2x 16 Core AMD Opteron 6376 |
| cpuspecs = 2x 16 Core AMD Opteron 6376 |
||
| Line 23: | Line 23: | ||
=Install= |
=Install= |
||
| − | Do a basic |
+ | Do a basic Arch install with the hard drives software raided in RAID 6, all of the device as / |
| + | |||
| + | Make sure to install GRUB to ALL of the drives and NOT /dev/md0 - also protip: md0 is a partition, not strictly a block device, thus it does not need a mbr and you can mkfs.ext4 /dev/md0 without issue :D |
||
==Install Packages== |
==Install Packages== |
||
<pre style="background-color:#ffcccc"> |
<pre style="background-color:#ffcccc"> |
||
| − | + | pacman -Syu htop iotop iftop lm_sensors vim sudo qemu libvirtd screen rsync tmux p7zip |
|
</pre> |
</pre> |
||
| − | Change SSHd to use port 13699 |
+ | Change SSHd to use port 13699 in addition to port 22 (add a line below Port:22 with Port:13699). Edit |
<pre style="background-color:#ccccff"> |
<pre style="background-color:#ccccff"> |
||
| Line 37: | Line 39: | ||
</pre> |
</pre> |
||
| − | == |
+ | ==User Configuration== |
| + | Add all of the users to the libvirtd and kvm groups that plan to use the VM host |
||
| − | We beed to create a network bridge for the VM's to connect on. |
||
| + | ==Network Configuration== |
||
| − | <pre style="background-color:#ccccff"> |
||
| − | /etc/network/interfaces |
||
| − | </pre> |
||
| + | VM's require a network bridge in order to be able to communicate to the network. |
||
| − | <pre style="background-color:#ccffcc"> |
||
| − | source /etc/network/interfaces.d/* |
||
| − | # The loopback network interface |
||
| − | auto lo |
||
| − | iface lo inet loopback |
||
| − | # Auto Configure eth0 and eth1 (our two NIC's - the ethernet over the usb ports is actually a IPMI port) |
||
| − | auto eth0 |
||
| − | auto eth1 |
||
| − | |||
| − | # Bridge, set the IP here and bridge all the defined ports for communication |
||
| − | auto br0 |
||
| − | iface br0 inet static |
||
| − | address 128.153.145.42 |
||
| − | netmask 255.255.254.0 |
||
| − | gateway 128.153.144.1 |
||
| − | bridge_ports all |
||
| − | </pre> |
||
=Virtual Machine Core Allocation= |
=Virtual Machine Core Allocation= |
||
| Line 123: | Line 107: | ||
Good to note is that there are 3 Ethernet ports. Two of them are NIC's (the pair right next to one another) and the one over the USB port is IPMI over Ethernet and has no connection to the kernel. Remote IPMI is disabled, so don't even bother with this. |
Good to note is that there are 3 Ethernet ports. Two of them are NIC's (the pair right next to one another) and the one over the USB port is IPMI over Ethernet and has no connection to the kernel. Remote IPMI is disabled, so don't even bother with this. |
||
| + | |||
| + | = Kernel Jankiness = |
||
| + | |||
| + | Under a Debian or Ubuntu kernel (even when custom compiled for the system by hand), shit goes haywire when you use GDB on this particular hardware configuration, for no reason. Thus, Arch Linux is the solution (for no apparent reason besides not having the symptoms) |
||
| + | |||
| + | Basically, the symptoms are that when debugging any process, we discovered that running si will cause the instruction pointer to rapidly become close to or become zero. We have no idea why this happens, but it's bad since it crashes the kernel. This includes if you are inside of a VM which is scary shit. Perhaps a CVE is in the works... |
||
Revision as of 21:36, 31 January 2017
| Hydra | |
| IP Address(es): | 128.153.145.42 |
| Contact Person: | Jared Dunbar & James Bruska |
| Last Update: | Spring 2017 |
| Services: | VM Host |
| Hostname: | hydra.cslabs.clarkson.edu |
| Operating system: | Arch Linux |
| NIC 1: | Clarkson Network MAC: 0c:c4:7a:c6:e7:88 IP: 128.153.145.42 |
| CPU: | 2x 16 Core AMD Opteron 6376 |
| RAM: | 4x 16GB DDR3-1600 ECC |
Hydra is our newest VM host (as of September 2016)
Contents
Install
Do a basic Arch install with the hard drives software raided in RAID 6, all of the device as /
Make sure to install GRUB to ALL of the drives and NOT /dev/md0 - also protip: md0 is a partition, not strictly a block device, thus it does not need a mbr and you can mkfs.ext4 /dev/md0 without issue :D
Install Packages
pacman -Syu htop iotop iftop lm_sensors vim sudo qemu libvirtd screen rsync tmux p7zip
Change SSHd to use port 13699 in addition to port 22 (add a line below Port:22 with Port:13699). Edit
/etc/ssh/sshd_config
User Configuration
Add all of the users to the libvirtd and kvm groups that plan to use the VM host
Network Configuration
VM's require a network bridge in order to be able to communicate to the network.
Virtual Machine Core Allocation
These are cores that are allocated to VM's for efficiency.
virsh vcpupin <guestname> <local-core> <host core>
Host cores are between 0 and 31 and local cores are from 0 to n-1 cores
You must run it for each core to each host core
| CPU CORE | ALLOCATION |
|---|---|
| 0 | SYSTEM - DO NOT ALLOCATE |
| 1 | Keyserver, scheduler, honeypot |
| 2 | VRkour, hunterGit |
| 3 | comm, hwiki, Turist |
| 4, 5 | cacti |
| 6, 7 | gmx |
| 24 to 31 | colona-evolutionary-cpu |
Free cores:
6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23
Hardware Information
IMPORTANT - DO NOT BUMP OR JOSTLE CASE
The CPU0 heat sink cooler is not very well mounted. It is mounted properly, however if the server is bumped too hard, the cooler will pop off. Treat the server with extreme care ALWAYS
- 2x 16 Core AMD Opeteron 6376
- 4x 16GB DDR3-1600 ECC RAM
- 5x 2TB Segate Constelation HDD in RAID 6
- 2x Noctua CPU Coolers
- Super Micro MBD-H8DG6-F-0 Motherboard EATX
- Silver Stone 550 Watt 80+ Platinum Power Supply
- Roswell 4u Server Chassis
Good to note is that there are 3 Ethernet ports. Two of them are NIC's (the pair right next to one another) and the one over the USB port is IPMI over Ethernet and has no connection to the kernel. Remote IPMI is disabled, so don't even bother with this.
Kernel Jankiness
Under a Debian or Ubuntu kernel (even when custom compiled for the system by hand), shit goes haywire when you use GDB on this particular hardware configuration, for no reason. Thus, Arch Linux is the solution (for no apparent reason besides not having the symptoms)
Basically, the symptoms are that when debugging any process, we discovered that running si will cause the instruction pointer to rapidly become close to or become zero. We have no idea why this happens, but it's bad since it crashes the kernel. This includes if you are inside of a VM which is scary shit. Perhaps a CVE is in the works...
