COSI Linux Build

From CSLabsWiki

This page summarizes how to set up the COSI Lab Build, and describes what procedures are in place to maintain it.

The Lab Build is currently based off of Ubuntu 9.04 (Jaunty Jackalope). It employs a `metapackage' repository hosted in COSI to keep track of all packages that should be installed. This way, when a user requests that a new package be installed, it is easy to update all of the machines - just add the requested package to the metapackage, which all lab build computers will update from overnight. The metapackage will be discussed in more detail later on.

Contents 1 Project Members 1.1 Current 1.2 Past 2 Requests 2.1 Pending 2.2 Finished 3 Lab Build Configuration 3.1 Initial Setup 3.2 Scripts 3.2.1 cleardesktop.sh 3.2.2 metapackage_upgrade.sh 3.2.3 net_setup.sh 3.2.4 hostname_setup.sh 3.2.5 mac_addresses.csv 4 The Metapackage 4.1 Overview 4.1.1 Benefits 4.1.2 Drawbacks 4.2 Setup 4.2.1 Creating a Metapackage 4.2.2 Creating a Repository

Project Members

Current

• Mark Platek

Past

• Chris Peterman
• Michael McKenna-Mattiaccio

Requests

Enter requests for the Lab Build here. If the request is available through apt, it will be added to the metapackage and the package will show up the next day if the machine is left running overnight. If the request is not available as a package, it will be added in the next ghost cycle. Please leave your name when you make a request. Alternatively, you can email the maintainer.

Pending

Please list requests here.

Finished

Note: items in this category have been added to the metapackage, which the build updates from at 4:00 AM. So, packages listed here might not appear right away.

• jEdit, apcalc, ant, antlr added 2009-10-31 (Sam Payson)

Lab Build Configuration

Initial Setup

Here are the steps to setting up the lab build. I will assume that the metapackage is already set up and hosted in a repository.

• Boot a single machine using the standard Ubuntu 9.04 liveCD. Install to hard drive, creating a 100M /boot partition (I used ext2), 1G of swap, and allocate the rest of the disk as / (I used ext4).
• Create user csguest with password cspassword; the `root' user will be created later. Complete any post-installation administrative tasks as required by the installer, such as setting the timezone. Set the hostname as cosi-<machine number>, like they're labeled. Get to an installed system in ready state at the desktop.
• Modify the file /etc/apt/sources.list. This is apt's repository configuration file. It is to be changed such that apt downloads packages from Mirror. Perform the following steps:
  • Do a search/replace: replace all instances of http://us.archive.ubuntu.com with http://mirror.clarkson.edu. Also replace http://security.ubuntu.com with http://mirror.clarkson.edu.
  • Uncomment the `universe' and `metaverse' repositories if they aren't already.
  • Add the line

    deb http://lab-build.cslabs.clarkson.edu/apt jaunty main

    The URL should point the the current location that hosts the metapackage.
• Run

  sudo apt-get update

  to update apt's list of installable packages. You can check for errors in sources.list by looking over the output.
• Run

  sudo apt-get -y upgrade

  to install the latest version of all default packages. Make sure everything is upgraded to the latest version before continuing.
• Run

  sudo apt-get -y install cosi-metapackage

  to install all packages specified by the metapackage. This will take a while! Sun Java forces you to accept a license agreement before it's installed, so you'll have to hang around at least until that point. After that, the process should be automatic, so don't bother hanging around waiting for it. The metapackage will also perform limited configuration steps (see postinst).
• Install programs that can't be handled by the metapackage.
  • Install Alloy Analyzer. Go to http://alloy.mit.edu/alloy4/ and download the latest .jar file for Alloy Analyzer. Store this .jar file in /usr/bin/ and add a launcher to the gnome applications list that executes

    java -jar /usr/bin/alloy4.jar

• Make configuration changes that can't be performed by postinst
  • The root user can be enabled from System->Administration->Users and Groups. Also create administrative user csadmin (be sure to add csadmin to group `admin') and restrict csguest.
  • Add script metapackage_upgrade.sh to crontab. Set it to run daily at an appropriate hour. Be sure to create empty file /root/bin/postinst_check.
  • Add script cleardesktop.sh to crontab. Set it to run daily at an appropriate hour.
  • Add script hostname_setup.sh to /etc/rc.local. It will run at every boot. Verify pathnames.
  • Set up subversion's .config file so that passwords aren't stored:

[auth]
store-passwords = no
store-auth-creds = no

• • Enable double-sided printing: System --> Administration --> Printing
    Installable Options: Check the box for "Duplex Unit"
    Printer Options: Under General, set 2-Sided Printing to Long-Edge Binding
  • Remove the irritating console beep:

    echo blacklist pcspkr | sudo tee -a /etc/modprobe.d/blacklist.conf > /dev/null

  • Set Firefox homepage to http://www.google.com and set it to ask the user to clear their session data when Firefox is closed.
  • Disable all sounds: System->Preferences->Sound->Sounds Tab; uncheck "Play alerts and sound effects". Also disable the Gnome Login Sound from System->System->Startup Applications. And, System->Administration->Login Window->Accessibility Tab; uncheck "Login screen ready".
  • Enable automatic login of csguest.
  • Finally, set up the desktop (widgets, stuff in the taskbars, etc) as you see fit.
• When the build is fully set up and configured, use clonezilla to create an image to ghost from. Leave it on a server, then use the clonezilla livecd to ghost out, reading from the server. Remove the file /etc/udev/rules.d/70-persistent-net.rules before making the initial image that will be cloned from.

Scripts

This section will contain scripts useful to the Lab Build.

cleardesktop.sh

This moves all files in /home/csguest/Desktop to /home/csguest/cleandesktop/[date]. After a week, the files will be deleted.

#!/bin/bash

# deletes any directory that's older than the 7th oldest directory present (in /home/csguest/cleandesktop)
removedirs()
{
        mkdir /tmp/cleandesktop
        DIRS=`ls /home/csguest/cleandesktop/ -xr -w 10000 | awk '{print $1 " "  $2 " " $3 " " $4 " " $5 " " $6 " " $7}'`
        for dir in $DIRS
        do
                mv /home/csguest/cleandesktop/$dir /tmp/cleandesktop
        done

        rm -rf /home/csguest/cleandesktop/*

        for dir in $DIRS
        do
                mv /tmp/cleandesktop/$dir /home/csguest/cleandesktop/
        done
}

# makes the backup directory containing all files on the Desktop
move_desktop_files()
{
        DATE=`date +%Y-%m-%d`
        mkdir /home/csguest/cleandesktop/$DATE
        mv /home/csguest/Desktop/* /home/csguest/cleandesktop/$DATE/
}

# ask the user if they want to stop the script from running
`zenity --question --timeout 60 --title "Danger, Will Robinson!" --text "The Desktop cleaning script is about to run. When it does, all files on the desktop will be moved to /home/csguest/cleandesktop/[today\'s date]. \n\n If you want to STOP the script and leave files on the desktop, hit \'OK\'."`

if [ $? == 0 ]; then
        DATE=`date +%Y-%m-%d`
        # not strictly necessary to even create this directory, but it will keep things neat, so
        # there will always be a week's worth of directories even if some are empty
        mkdir /home/csguest/cleandesktop/$DATE
        removedirs
        exit
fi

# perform the desktop cleaning
move_desktop_files
removedirs

exit

metapackage_upgrade.sh

This updates the system. It is currently configured to run at login, and daily at 4:00 AM.

#!/bin/bash

# this script simply updates apt's cache before upgrading
# purging the metapackage doesn't remove anything but the metapackage! it's necessary to do this in order to update the postinst script

/usr/bin/apt-get -y update
/usr/bin/apt-get -y purge cosi-metapackage
/usr/bin/apt-get -y --force-yes install cosi-metapackage
/usr/bin/apt-get -y upgrade

exit

net_setup.sh

This sets static IPs based upon a table of known MAC addresses. It also sets the hostname.

#!/bin/bash

my_mac=`ifconfig | grep eth0 | awk '{print $5}'`
echo "detected mac $my_mac"

my_hostname=`grep $my_mac mac_addresses.csv | awk -F, '{print $1}'`
echo "my hostname should be set to $my_hostname"


my_eth0_ip=`grep ^$my_hostname mac_addresses.csv | awk -F, '{print $4}'`
my_eth1_ip=`grep ^$my_hostname mac_addresses.csv | awk -F, '{print $5}'`
my_eth0_mac=`grep ^$my_hostname mac_addresses.csv | awk -F, '{print $2}'`
my_eth1_mac=`grep ^$my_hostname mac_addresses.csv | awk -F, '{print $3}'`


cat > /etc/hosts <<EOF
127.0.0.1       localhost
127.0.1.1       $my_hostname
EOF

cat > /etc/hostname <<EOF
$my_hostname
EOF

cat > /etc/network/interfaces <<EOF
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
address $my_eth0_ip
gateway 128.153.144.1
netmask 255.255.254.0
nameserver 128.153.0.254
search clarkson.edu

auto eth1
iface eth1 inet static
address $my_eth1_ip
gateway 192.168.44.1
netmask 255.255.254.0

EOF

cat > /etc/udev/rules.d/70-persistent-net.rules <<EOF

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="$my_eth0_mac", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="$my_eth1_mac", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

EOF

hostname_setup.sh

This is a subset of the functionality provided by net_setup.sh. It uses DHCP to acquire IPs and sets only the hostname (based on the MAC address).

#!/bin/bash

macaddrs="/root/bin/mac_addresses.csv"

my_mac=`/sbin/ifconfig -a | grep eth0 | awk '{print $5}'`
echo "detected mac $my_mac"

my_hostname=`grep $my_mac $macaddrs | awk -F, '{print $1}'`
echo "my hostname should be set to $my_hostname"

cat > /etc/hosts <<EOF
127.0.0.1       localhost
127.0.1.1       $my_hostname
EOF

cat > /etc/hostname <<EOF
$my_hostname
EOF

exit

mac_addresses.csv

cosi-01,00:11:25:f6:15:22,00:14:6c:2e:49:e0,128.153.144.141,192.168.44.1
cosi-02,00:11:25:f6:5c:13,00:0f:b5:fd:a4:00,128.153.144.142,192.168.44.2
cosi-03,00:11:25:f6:52:95,00:0f:b5:fe:4b:e5,128.153.144.143,192.168.44.3
cosi-04,00:11:25:f6:5d:c9,00:0f:b5:fb:77:da,128.153.144.144,192.168.44.4
cosi-05,00:11:25:f6:70:7a,00:0f:b5:8f:9d:67,128.153.144.145,192.168.44.5
cosi-06,00:11:25:f6:5d:7c,00:0f:b5:fb:9f:81,128.153.144.146,192.168.44.6
cosi-07,00:11:25:f6:70:8c,00:0f:b5:f8:75:db,128.153.144.147,192.168.44.7
cosi-08,00:11:25:f6:67:9c,00:0f:b5:47:a7:d5,128.153.144.148,192.168.44.8
cosi-09,00:11:25:f6:2f:c1,00:14:6c:2e:47:df,128.153.144.149,192.168.44.9
cosi-10,00:11:25:f6:5b:2e,00:14:6c:2e:49:e0,128.153.144.150,192.168.44.10
vr-cosi-01,00:1a:a0:a9:62:88,00:18:4d:f0:0f:a4,128.153.144.151,192.168.44.11
vr-cosi-02,00:1a:a0:a9:f4:f1,00:18:4d:f0:0e:43,128.153.144.152,192.168.44.12
vr-cosi-03,00:1a:a0:a9:f2:81,00:18:4d:f0:19:07,128.153.144.153,192.168.44.13
vr-cosi-04,00:1a:a0:a9:f5:32,00:18:4d:f0:0a:4a,128.153.144.154,192.168.44.14
vr-cosi-05,00:1a:a0:a9:f5:0a,00:18:4d:f0:11:17,128.153.144.155,192.168.44.15
vr-cosi-06,00:1a:a0:a9:60:e2,00:18:4d:f0:10:f5,128.153.144.156,192.168.44.16

The Metapackage

Overview

The metapackage is nothing but a Debian package (that is, it's installable with apt-get) that itself installs other packages. The idea is that, rather than expend lots of bandwidth and cause premature wear on hard drives by re-ghosting every time packages are added to the lab build, each machine can keep itself up to date by reinstalling a newer version of the metapackage (which itself can be easily updated to contain the new packages).

Benefits

• Ghosting takes time and puts strain on the hardware if performed frequently. Using a metapackage, each machine will only perform the minimum amount of work necessary to stay updated, and need not be taken out of commission while ghosting occurs.
• The metapackage itself is a convenient list of the software composing the lab build.
• A lab build VM need not exist to ghost from. This leaves more room free for other projects requiring VMs.

Drawbacks

• The machines will still have to be ghosted if there are non-trivial configuration changes.
• Some software is not distributed as a Debian package and cannot be included in the metapackage. Such software must be handled on an individual basis and ghosted out if a .deb cannot be created.

Setup

There are two main components of the metapackage system. First is the metapackage itself, which will be created with dpkg. Second is the repository, which will be created and managed with reprepro.

Creating a Metapackage

The general idea is to create a directory structure that can be understood by dpkg, then to write the control (and postinst) files to describe the metapackage. Finally, dpkg will be used to create a .deb file.

• Create the following directory structure:

  ../cosi-metapackage-<version>/DEBIAN

• Inside the DEBIAN directory, create file control. This textfile will describe the metapackage. A sample control file is given below:

Package: cosi-metapackage
Version: <version number>
Section: main
Priority: optional
Architecture: i386
Depends: list, of, packages, to, install, here
Conflicts: packages, to, be, removed
Replaces: packages, to, be, removed
Maintainer: <name>
Description: Text describing the package

• To add packages to the metapackage, list them under `Depends'. To ensure that certain packages are not installed, or to remove them, add the package name to BOTH `Conflicts' and `Replaces'.
• Create file (postinst) in the DEBIAN directory. This file will be executed as a script after all of the packages in `Depends' have been installed. It will be used to configure packages that might not be ready-to-run right after being installed. If any packages need this extra step, add the relevant bash commands to this script.
• Return to the directory two levels above DEBIAN. If you don't, dpkg will still work if you give it the relative pathname to directory cosi-metapackage-<version>/.
• Run

  dpkg --build cosi-metapackage-<version>/ cosi-metapackage-<version>.deb

  which will create in the working directory the .deb file to added to the repository.

Creating a Repository

• Create a directory for the repository to use. Inside it, create directory conf. Inside that, create file distributions.
• Fill out the distributions file with information defining the repository. A sample is given below:

Origin: <maintainer name>
Label: COSI Lab Build repository
Suite: stable
Codename: jaunty
Version: 9.04
Architectures: i386 amd64 all
Components: main
Description: Text description

• The `Components' field must match the metapackage .deb's control file, otherwise you won't be able to add it to the repository.
• From the base repository directory, run

  sudo reprepro includedeb jaunty /path/to/cosi-metapackage<version>.deb

• If all goes well, you can now point the sources.list file to the base repository directory, and be able to install the metapackage using apt.
• If a package must be removed from the repository, run

  sudo reprepro remove jaunty <packagename>