COSI Linux Build

From CSLabsWiki
Revision as of 10:12, 3 February 2010 by Mccarrms (talk | contribs) (Pending)

This page summarizes how to set up the COSI Lab Build, and describes what procedures are in place to maintain it.

The Lab Build is currently based off of Ubuntu 9.04 (Jaunty Jackalope). It employs a `metapackage' repository hosted in COSI to keep track of all packages that should be installed. This way, when a user requests that a new package be installed, it is easy to update all of the machines - just add the requested package to the metapackage, which all lab build computers will update from overnight. The metapackage will be discussed in more detail later on.

Project Members

Current

Past

Requests

Enter requests for the Lab Build here. If the request is available through apt, it will be added to the metapackage and the package will show up the next day if the machine is left running overnight. If the request is not available as a package, it will be added in the next ghost cycle. Please leave your name when you make a request. Alternatively, you can email the maintainer.

Pending

Please list requests here.

  • Java runtime in Firefox --Matt 09:12, 3 February 2010 (EST)

Finished

Note: items in this category have been added to the metapackage, which the build updates from at 4:00 AM. So, packages listed here might not appear right away.

  • jEdit, apcalc, ant, antlr added 2009-10-31 (Sam Payson)

Next Version

Here is a list of features, etc. that I'm going to add to the next lab build. Please add feature requests to the `Requests' section.

  • rebase to Ubuntu 9.10 (latest), here is a list of packages that cause problems
    • dontzap (solution: best solution is to add merge line to the fdi)
    • libopenmpi1 (maybe this is just a dependency?)
    • libcrypto++7 (I think this is also a dep)
    • flashplugin-installer (solution: install package ubuntu-restricted-extras)
  • ensure that all cron jobs run properly (cron.allow? why would cron be disabled by default?)
  • add kiosckar support (would be for vr-cosi machines, can this be done properly?)
  • set static IPs for each machine using list of MACs
  • set up a clonezilla server (drbl) for ease of cloning
  • Add bookmark for print in the browser
  • Change terminal background color to standard black
  • add program: yices

Lab Build Configuration

Initial Setup

Here are the steps to setting up the lab build. I will assume that the metapackage is already set up and hosted in a repository.

  • Boot a single machine using the standard Ubuntu 9.04 liveCD. Install to hard drive, creating a 100M /boot partition (I used ext2), 1G of swap, and allocate the rest of the disk as / (I used ext4).
  • Create user csguest with password cspassword; the `root' user will be created later. Complete any post-installation administrative tasks as required by the installer, such as setting the timezone. Set the hostname as cosi-<machine number>, like they're labeled. Get to an installed system in ready state at the desktop.
  • Modify the file /etc/apt/sources.list. This is apt's repository configuration file. It is to be changed such that apt downloads packages from Mirror. Perform the following steps:
  • Run
    sudo apt-get update
    to update apt's list of installable packages. You can check for errors in sources.list by looking over the output.
  • Run
    sudo apt-get -y upgrade
    to install the latest version of all default packages. Make sure everything is upgraded to the latest version before continuing.
  • Run
    sudo apt-get -y install cosi-metapackage
    to install all packages specified by the metapackage. This will take a while! Sun Java forces you to accept a license agreement before it's installed, so you'll have to hang around at least until that point. After that, the process should be automatic, so don't bother hanging around waiting for it. The metapackage will also perform limited configuration steps (see postinst).
  • Install programs that can't be handled by the metapackage.
    • Install Alloy Analyzer. Go to http://alloy.mit.edu/alloy4/ and download the latest .jar file for Alloy Analyzer. Store this .jar file in /usr/bin/ and add a launcher to the gnome applications list that executes
      java -jar /usr/bin/alloy4.jar
  • Make configuration changes that can't be performed by postinst
    • The root user can be enabled from System->Administration->Users and Groups. Also create administrative user csadmin (be sure to add csadmin to group `admin') and restrict csguest.
    • Add script metapackage_upgrade.sh to crontab. Set it to run daily at an appropriate hour. Be sure to create empty file /root/bin/postinst_check.
    • Add script cleardesktop.sh to crontab. Set it to run daily at an appropriate hour.
    • Add script hostname_setup.sh to /etc/rc.local. It will run at every boot. Verify pathnames.
    • Set up subversion's .config file so that passwords aren't stored:
[auth]
store-passwords = no
store-auth-creds = no
    • Enable double-sided printing: System --> Administration --> Printing
      Installable Options: Check the box for "Duplex Unit"
      Printer Options: Under General, set 2-Sided Printing to Long-Edge Binding
    • Remove the irritating console beep:
      echo blacklist pcspkr | sudo tee -a /etc/modprobe.d/blacklist.conf > /dev/null
    • Set Firefox homepage to http://www.google.com and set it to ask the user to clear their session data when Firefox is closed.
    • Disable all sounds: System->Preferences->Sound->Sounds Tab; uncheck "Play alerts and sound effects". Also disable the Gnome Login Sound from System->System->Startup Applications. And, System->Administration->Login Window->Accessibility Tab; uncheck "Login screen ready".
    • Enable automatic login of csguest.
    • Finally, set up the desktop (widgets, stuff in the taskbars, etc) as you see fit.
  • When the build is fully set up and configured, use clonezilla to create an image to ghost from. Leave it on a server, then use the clonezilla livecd to ghost out, reading from the server. Remove the file /etc/udev/rules.d/70-persistent-net.rules before making the initial image that will be cloned from.

Scripts

This section will contain scripts useful to the Lab Build.

cleardesktop.sh

This moves all files in /home/csguest/Desktop to /home/csguest/cleandesktop/[date]. After a week, the files will be deleted. <source lang="bash">

  1. !/bin/bash
  1. deletes any directory that's older than the 7th oldest directory present (in /home/csguest/cleandesktop)

removedirs() {

       mkdir /tmp/cleandesktop
       DIRS=`ls /home/csguest/cleandesktop/ -xr -w 10000 | awk '{print $1 " "  $2 " " $3 " " $4 " " $5 " " $6 " " $7}'`
       for dir in $DIRS
       do
               mv /home/csguest/cleandesktop/$dir /tmp/cleandesktop
       done
       rm -rf /home/csguest/cleandesktop/*
       for dir in $DIRS
       do
               mv /tmp/cleandesktop/$dir /home/csguest/cleandesktop/
       done

}

  1. makes the backup directory containing all files on the Desktop

move_desktop_files() {

       DATE=`date +%Y-%m-%d`
       mkdir /home/csguest/cleandesktop/$DATE
       mv /home/csguest/Desktop/* /home/csguest/cleandesktop/$DATE/

}

  1. ask the user if they want to stop the script from running

`zenity --question --timeout 60 --title "Danger, Will Robinson!" --text "The Desktop cleaning script is about to run. When it does, all files on the desktop will be moved to /home/csguest/cleandesktop/[today\'s date]. \n\n If you want to STOP the script and leave files on the desktop, hit \'OK\'."`

if [ $? == 0 ]; then

       DATE=`date +%Y-%m-%d`
       # not strictly necessary to even create this directory, but it will keep things neat, so
       # there will always be a week's worth of directories even if some are empty
       mkdir /home/csguest/cleandesktop/$DATE
       removedirs
       exit

fi

  1. perform the desktop cleaning

move_desktop_files removedirs

exit

</source>

metapackage_upgrade.sh

This updates the system. It is currently configured to run at login, and daily at 4:00 AM. <source lang="bash">

  1. !/bin/bash
  1. this script simply updates apt's cache before upgrading
  2. purging the metapackage doesn't remove anything but the metapackage! it's necessary to do this in order to update the postinst script

/usr/bin/apt-get -y update /usr/bin/apt-get -y purge cosi-metapackage /usr/bin/apt-get -y --force-yes install cosi-metapackage /usr/bin/apt-get -y upgrade

exit </source>

net_setup.sh

This sets static IPs based upon a table of known MAC addresses. It also sets the hostname. <source lang="bash">

  1. !/bin/bash

my_mac=`ifconfig | grep eth0 | awk '{print $5}'` echo "detected mac $my_mac"

my_hostname=`grep $my_mac mac_addresses.csv | awk -F, '{print $1}'` echo "my hostname should be set to $my_hostname"


my_eth0_ip=`grep ^$my_hostname mac_addresses.csv | awk -F, '{print $4}'` my_eth1_ip=`grep ^$my_hostname mac_addresses.csv | awk -F, '{print $5}'` my_eth0_mac=`grep ^$my_hostname mac_addresses.csv | awk -F, '{print $2}'` my_eth1_mac=`grep ^$my_hostname mac_addresses.csv | awk -F, '{print $3}'`


cat > /etc/hosts <<EOF 127.0.0.1 localhost 127.0.1.1 $my_hostname EOF

cat > /etc/hostname <<EOF $my_hostname EOF

cat > /etc/network/interfaces <<EOF auto lo iface lo inet loopback

auto eth0 iface eth0 inet static address $my_eth0_ip gateway 128.153.144.1 netmask 255.255.254.0 nameserver 128.153.0.254 search clarkson.edu

auto eth1 iface eth1 inet static address $my_eth1_ip gateway 192.168.44.1 netmask 255.255.254.0

EOF

cat > /etc/udev/rules.d/70-persistent-net.rules <<EOF

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="$my_eth0_mac", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="$my_eth1_mac", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

EOF </source>

hostname_setup.sh

This is a subset of the functionality provided by net_setup.sh. It uses DHCP to acquire IPs and sets only the hostname (based on the MAC address). <source lang="bash">

  1. !/bin/bash

macaddrs="/root/bin/mac_addresses.csv"

my_mac=`/sbin/ifconfig -a | grep eth0 | awk '{print $5}'` echo "detected mac $my_mac"

my_hostname=`grep $my_mac $macaddrs | awk -F, '{print $1}'` echo "my hostname should be set to $my_hostname"

cat > /etc/hosts <<EOF 127.0.0.1 localhost 127.0.1.1 $my_hostname EOF

cat > /etc/hostname <<EOF $my_hostname EOF

exit </source>

mac_addresses.csv

cosi-01,00:11:25:f6:15:22,00:14:6c:2e:49:e0,128.153.144.141,192.168.44.1
cosi-02,00:11:25:f6:5c:13,00:0f:b5:fd:a4:00,128.153.144.142,192.168.44.2
cosi-03,00:11:25:f6:52:95,00:0f:b5:fe:4b:e5,128.153.144.143,192.168.44.3
cosi-04,00:11:25:f6:5d:c9,00:0f:b5:fb:77:da,128.153.144.144,192.168.44.4
cosi-05,00:11:25:f6:70:7a,00:0f:b5:8f:9d:67,128.153.144.145,192.168.44.5
cosi-06,00:11:25:f6:5d:7c,00:0f:b5:fb:9f:81,128.153.144.146,192.168.44.6
cosi-07,00:11:25:f6:70:8c,00:0f:b5:f8:75:db,128.153.144.147,192.168.44.7
cosi-08,00:11:25:f6:67:9c,00:0f:b5:47:a7:d5,128.153.144.148,192.168.44.8
cosi-09,00:11:25:f6:2f:c1,00:14:6c:2e:47:df,128.153.144.149,192.168.44.9
cosi-10,00:11:25:f6:5b:2e,00:14:6c:2e:49:e0,128.153.144.150,192.168.44.10
vr-cosi-01,00:1a:a0:a9:62:88,00:18:4d:f0:0f:a4,128.153.144.151,192.168.44.11
vr-cosi-02,00:1a:a0:a9:f4:f1,00:18:4d:f0:0e:43,128.153.144.152,192.168.44.12
vr-cosi-03,00:1a:a0:a9:f2:81,00:18:4d:f0:19:07,128.153.144.153,192.168.44.13
vr-cosi-04,00:1a:a0:a9:f5:32,00:18:4d:f0:0a:4a,128.153.144.154,192.168.44.14
vr-cosi-05,00:1a:a0:a9:f5:0a,00:18:4d:f0:11:17,128.153.144.155,192.168.44.15
vr-cosi-06,00:1a:a0:a9:60:e2,00:18:4d:f0:10:f5,128.153.144.156,192.168.44.16

The Metapackage

Overview

The metapackage is nothing but a Debian package (that is, it's installable with apt-get) that itself installs other packages. The idea is that, rather than expend lots of bandwidth and cause premature wear on hard drives by re-ghosting every time packages are added to the lab build, each machine can keep itself up to date by reinstalling a newer version of the metapackage (which itself can be easily updated to contain the new packages).

Benefits

  • Ghosting takes time and puts strain on the hardware if performed frequently. Using a metapackage, each machine will only perform the minimum amount of work necessary to stay updated, and need not be taken out of commission while ghosting occurs.
  • The metapackage itself is a convenient list of the software composing the lab build.
  • A lab build VM need not exist to ghost from. This leaves more room free for other projects requiring VMs.

Drawbacks

  • The machines will still have to be ghosted if there are non-trivial configuration changes.
  • Some software is not distributed as a Debian package and cannot be included in the metapackage. Such software must be handled on an individual basis and ghosted out if a .deb cannot be created.

Setup

There are two main components of the metapackage system. First is the metapackage itself, which will be created with dpkg. Second is the repository, which will be created and managed with reprepro.

Creating a Metapackage

The general idea is to create a directory structure that can be understood by dpkg, then to write the control (and postinst) files to describe the metapackage. Finally, dpkg will be used to create a .deb file.

  • Create the following directory structure:
    ../cosi-metapackage-<version>/DEBIAN
  • Inside the DEBIAN directory, create file control. This textfile will describe the metapackage. A sample control file is given below:
Package: cosi-metapackage
Version: <version number>
Section: main
Priority: optional
Architecture: i386
Depends: list, of, packages, to, install, here
Conflicts: packages, to, be, removed
Replaces: packages, to, be, removed
Maintainer: <name>
Description: Text describing the package
  • To add packages to the metapackage, list them under `Depends'. To ensure that certain packages are not installed, or to remove them, add the package name to BOTH `Conflicts' and `Replaces'.
  • Create file (postinst) in the DEBIAN directory. This file will be executed as a script after all of the packages in `Depends' have been installed. It will be used to configure packages that might not be ready-to-run right after being installed. If any packages need this extra step, add the relevant bash commands to this script.
  • Return to the directory two levels above DEBIAN. If you don't, dpkg will still work if you give it the relative pathname to directory cosi-metapackage-<version>/.
  • Run
    dpkg --build cosi-metapackage-<version>/ cosi-metapackage-<version>.deb
    which will create in the working directory the .deb file to added to the repository.

Creating a Repository

  • Create a directory for the repository to use. Inside it, create directory conf. Inside that, create file distributions.
  • Fill out the distributions file with information defining the repository. A sample is given below:
Origin: <maintainer name>
Label: COSI Lab Build repository
Suite: stable
Codename: jaunty
Version: 9.04
Architectures: i386 amd64 all
Components: main
Description: Text description
  • The `Components' field must match the metapackage .deb's control file, otherwise you won't be able to add it to the repository.
  • From the base repository directory, run
    sudo reprepro includedeb jaunty /path/to/cosi-metapackage<version>.deb
  • If all goes well, you can now point the sources.list file to the base repository directory, and be able to install the metapackage using apt.
  • If a package must be removed from the repository, run
    sudo reprepro remove jaunty <packagename>