Creating an image and exporting it over AoE
From CSLabsWiki
 |
This article discusses a deprecated technology or practice. Reason: Major infrastructure changes took place in Spring 2009. See Create a New VM for the current process. Visit the talk page for more information. |
- See also: Image Creation Checklist
This how to summarizes how to create a virtual machine image and export it over AoE. It is assumed that these steps are being preformed on the setup involving dns, Animal, the Blade Center, and the IBM RAID Storage. To follow these steps, you need root/sudo access on dns, Animal, and the Dom0.
First, a virtual machine image is made and changes to the network and xen configurations are made. It is then exported over the network. It can then be mounted on the dom0, at which point it can be configured. The virtual machine is started, and some final changes are made to the configuration.
Contents |
Editing the Configuration Scripts
This step is necessary to ensure that your virtual machine is correctly handled and exported.
Editing the Networking Scripts
On dns, you need to edit sr_hosts & cu_hosts in /root/scripts/ adding a line in the format:
<imagename> <IP Address>
You then need to run:
dnsupdate
Editing the Xen Config and AoE Exports list
On Animal, you need to edit masterlist in /root/scripts/ adding a line in the format:
<imagename> eX.Yp2 eX.Yp1 <ram> <External MAC Address> <External Enabled> <Server Room MAC Address> <Server Room Enabled> <Uses AoE> <imagename>.disk
For most virtual machines, the following would be sufficient:
<imagename> eX.Yp2 eX.Yp1 <ram> <External MAC Address> 1 <Server Room MAC Address> 1 1 <imagename>.disk
For a full explanation of this config file, see the separate article Animal's Confgen Script.
To determine the values for X(shelf) and Y(slot), use the next available shelf/slot combination. There is a 0-15 range for slots, so the logical order to assign them in is:
- 1,1
- 1,2
- ...
- 1,15
- 2,0
- 2,1
- ...
- 2,15
You then need to run:
/root/scripts/confgen.pl /root/scripts/masterlist /mnt/raidB/xenlib/conf
This script needs a bit of house cleaning. There isn't any error checking or validation so a simple blank line could break the scripts. Watch out ;-)
Creating the Image
Creation of the .disk file
The image creation occurs on Animal. Once you are logged in, you need to run the following command: (make sure there is not a .disk file with the same name as the image you're creating!):
# dd if=/dev/zero of=/mnt/raidB/xenlib/images/<imagename>.disk bs=1024k count=1 seek=4095
(In order to create an image larger than 4 GBs, you could increase the seek value, but for most images, 4 GB is plenty) You must then export the .disk file using the command:
# vbladed X Y eth0 /mnt/raidB/xenlib/images/<imagename>.disk
(where X and Y were the same chosen when editing the configuration.)
After all of these steps are completed, the file you initiated is being forwarded to all blades and therefore installing an operating system onto the file can be done from any blade, as long as you have root access on the Dom0 of that blade.
Resize an existing .disk file
Installing an operating system
These steps occur on the dom0 (one of the Blades).
Formating the Image
The first step is to partition and format the image. To partition the image, fdisk is used. Run the command:
# fdisk /dev/etherd/eX.Y
At the prompt, enter "n" for new partition, then enter "p" followed by "1" to make it the first primary partition. The default is correct for the next prompt, followed by "64" (to make the first partition 512MB). Now at the prompt again, enter "n" for new partition, then enter "p" followed by "2" to make it the second primary partition. The default is correct for the next two prompts. Now at the prompt for the third time, enter "w" to write the partition table and exit fdisk.
The next step is to format each of the partitions. To format the first partition as swap, run the command:
# mkswap /dev/etherd/eX.Yp1
To format the second partition as ext3, run the command:
# mkfs.ext3 /dev/etherd/eX.Yp2
Using debootstrap to install a Distribution
The next step is to install a distribution. At this time, this how to only covers Debian and Ubuntu.
First, mount the second partition using the following commands:
# mkdir /mnt/tmp
# mount /dev/etherd/eX.Yp2 /mnt/tmp
Then, run one of the following debootstrap commands:
For Debian Etch:
# debootstrap --arch amd64 etch /mnt/tmp http://ftp.us.debian.org/debian
For Ubuntu Gutsy:
# debootstrap --arch amd64 hardy /mnt/tmp http://mirror.sr/pub/ubuntu
or, if that doesn't work
# debootstrap --arch amd64 hardy /mnt/tmp http://archive.ubuntu.com/ubuntu
Configuring the Virtual Machine
With the partition still mounted, edit the following files:
- /mnt/tmp/etc/hostname
- The only contents of the file should be the name of your image
NEWVM=newvmname; sudo perl -pi -e "s/`hostname`/$NEWVM/g" /mnt/tmp/etc/hostname
- /mnt/tmp/etc/hosts
- If the machine is not on the server room network, the contents of this file should be:
127.0.0.1 localhost <image name>
NEWVM=newvmname; sudo echo "127.0.0.1 localhost $NEWVM" > /mnt/tmp/etc/hosts- If the machine is on the server room network, the contents of this file should be:
127.0.0.1 localhost <image name> <internal IP> <image name>.sr
- /mnt/tmp/etc/dhcp3/dhclient.conf
- If the server is on the server room network, uncomment the prepend domain-name-servers line and modify it so that it reads "prepend domain-name-servers 10.0.1.1;"
- /mnt/tmp/etc/fstab
- The contents of the file should be:
/dev/sda1 / ext3 defaults 0 1
/dev/sda2 swap swap defaults 0 0
proc /proc proc defaults 0 0
/dev/sda3 /lib/modules ext3 ro 0 0
- /mnt/tmp/etc/network/interfaces
- If the image has a static IP address, the file should look like:
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
address <static IP here>
netmask 255.255.255.0
gateway 128.153.145.1
auto eth1
iface eth1 inet dhcp
- Otherwise, the file should be:
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet dhcp
auto eth1
iface eth1 inet dhcp
- If the image only has access to the Clarkson Network, then
/etc/hosts.allowshould be modified to look like the following.
sshd : 128.153.145.0/24 : allow
sshd : ALL : deny
Then, unmount the second partition using:
# umount /mnt/tmp
# rmdir /mnt/tmp
And start the virtual machine using:
# xm create -c /xen/<imagename>
Once the virtual machine has started:
- Change the root password
- If the machine is running debian etch:
- Run # apt-get install libc6-xen sudo
- Reboot the image
- Create user accounts as necessary
- Install the SSH server
- Run # apt-get install openssh-server
- Edit /etc/ssh/sshd_config
- If your image is on the server room network (see the configuration section above), set the ListenAddress (ListenAddress <imagename>.sr)
- Disable root ssh (PermitRootLogin no)
- Install Emacs & Nano Editors
- Run # apt-get install emacs22-nox nano
- If you need web services, you can install them using # apt-get install apache2 php5 libapache2-mod-php5 mysql-server mysql-client php5-mysql
- Optionally setup LDAP following the guide here
Conclusion
The virtual machine should now be up and running. Don't forget to document the changes you made, and add to the List of VMs.
Notes
AoE in Virtual Machines
Because virtual machines mount their /lib/modules directory, modules cannot be loaded at the time /etc/modules is run. When /etc/fstab is reached, there is no way (with the current system) to have aoe running, so it is necessary to construct an rc.local file like the one below:
# Modules for AoE
modprobe aoe
modprobe dm-multipath
# Sleeping to allow aoe to detect all devices
sleep 5
mount -a
Troubleshooting
Setting locale failed errors
If you are getting errors that look like this:
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
then run the command locale-gen en_US.UTF-8 (assuming en_US.UTF-8 is the LANG that isn't being found).
