Difference between revisions of "Storage Setup Process"

From CSLabsWiki
(commit)
(commit)
Line 493: Line 493:
 
0 * * * * /usr/bin/updatedb
  
0 * * * * /usr/bin/updatedb
 
</pre>
  
</pre>
  +
  +
== Disable CTRL-ALT-DELETE ==
  +
Remove trap entry to prevent accidental reboots, then make changes active:
  +
<pre>
  +
sed -i 's/ca::ctrlaltdel:/#ca::ctrlaltdel:/g' /etc/inittab
  +
init q
  +
</pre>
  +
  +
   
 
= Network Configuration =
  
= Network Configuration =
Line 777: Line 786:
 
</pre>
  
</pre>
   
  +
= Mail and Logging =
   
  +
== Configure Aliases ==
−  
  +
Edit <tt>/etc/aliases</tt>:
−  
  +
<pre>
−  
−  
−  
−  
−  
−  
−  
−  
−  
−  
−  
−  
−  
−  
−
===Configured Aliases===
  
−
*Edited <code>/etc/aliases</code>
  
−
<code><pre>
  
 
#
  
#
 
# Aliases in this file will NOT be expanded in the header from
  
# Aliases in this file will NOT be expanded in the header from
Line 893: Line 887:
 
# Person who should get roots's mail
  
# Person who should get roots's mail
 
root: logwatch@cslabs.clarkson.edu
  
root: logwatch@cslabs.clarkson.edu
−
</pre></code>
 +
</pre>
  +
  +
With that config file in place, update the aliases:
  +
<pre>
  +
/usr/bin/newaliases
  +
</pre>
  +
  +
  +
  +
  +
  +
  +
  +
  +
  +
  +
  +
  +
  +
  +
  +
  +
  +
   
−
*Updated aliases
  
−
**<code>/usr/bin/newaliases</code>
  
   
−
===Disabled <code>CTRL-ALT-DELETE</code>===
  
−
*Removed trap entry to prevent accidental reboots
  
−
<code><pre>
  
−
sed -i 's/ca::ctrlaltdel:/#ca::ctrlaltdel:/g' /etc/inittab
  
−
</pre></code>
  
   
−
*Made Changes Active
  
−
<code><pre>
  
−
init q
  
−
</pre></code>
  
   
 
===Disabled Various Kernel Modules===
  
===Disabled Various Kernel Modules===

Revision as of 18:19, 15 November 2010

This page summarizes how Storage was set up in Fall 2010.

Installation

Operating system: CentOS 5.5 x86_64

Partition Scheme

Partition Scheme:

  • /dev/md0: 100MB, /boot, mdRAID 1
  • /dev/md1: 80GB, LVM (root_lvg), mdRAID 1
    • /dev/root_lvg/root_lv: 65GB, /
    • /dev/root_lvg/tmp_lv: 1GB, /tmp
    • /dev/root_lvg/var_lv: 4GB, /var
    • /dev/root_lvg/home_lv: 1GB, /home
    • /dev/root_lvg/swap_lv: 4GB, swap
  • /dev/md2: 3TB, /storage, mdRAID 10

Partition Types

The boot array is formatted as ext2, all other partitions except the storage array are formatted as ext3.

The storage partition is formatted as xfs, a robust filesystem that deals particularly well with large files. The filesystem was created with the following command: 

mkfs.xfs -d agcount=1000 -l size=128m /dev/md2

This creates an xfs filesystem with 1000 AGs (each of 3GB) and a log size of 128MB. Having many AGs makes the filesystem more parallelizable at cost of CPU cycles (of which we have plenty), and the higher-than-default log size increases the number of small writes that can take place quickly in memory and be committed asynchronously in contiguous bursts.

The filesystem is mounted in /etc/fstab with the following options: 

/dev/md2    /storage    xfs    defaults,logbufs=8,noatime    1 2

This mounts the filesystem with 8 log buffers, again increasing the number of transactions able to be performed in memory at once. Option 'noatime' causes inode access times to not be updated, increasing performance slightly. It's OK to do this since inode access times are not generally used.

In order to use xfs in CentOS 5.5, the following packages were installed: 

kmod-xfs
xfsdump
xfsprogs

Kickstart

This is the kickstart file appearing in /root after the install finished. There is a storage kickstart file living on Admin that was used for the netinstall. 

# Kickstart file automatically generated by anaconda.

install
url --url http://mirror.clarkson.edu/centos/5/os/x86_64/
lang en_US.UTF-8
keyboard us
network --device eth0 --bootproto static --ip 128.153.145.40 --netmask 255.255.255.0 --gateway 128.153.145.1 --nameserver 128.153.0.254,128.153.5.254 --hostname storage
network --device eth1 --bootproto static --ip 10.0.1.35 --netmask 255.255.255.0 --gateway 128.153.145.1 --nameserver 128.153.0.254,128.153.5.254 --hostname storage
network --device eth2 --bootproto static --ip 10.0.0.15 --netmask 255.255.255.0 --gateway 128.153.145.1 --nameserver 128.153.0.254,128.153.5.254 --hostname storage
rootpw --iscrypted $1$5UvTrOQ1$ttBrGsABSlAoVyxdlf4wg/
firewall --enabled --port=22:tcp
authconfig --enableshadow --enablemd5
selinux --permissive
timezone --utc America/New_York
bootloader --location=partition --driveorder=hda,hdb,sda,sdb,sdc,sdd,sde,sdf
# The following is the partition information you requested
# Note that any partitions you deleted are not expressed
# here so unless you clear all partitions first, this is
# not guaranteed to work
#part raid.1 --noformat --onpart hda1
#part raid.3 --noformat --onpart hdb1
#part raid.10 --noformat --onpart sdf1
#part raid.9 --noformat --onpart sde1
#part raid.8 --noformat --onpart sdd1
#part raid.7 --noformat --onpart sdc1
#part raid.6 --noformat --onpart sdb1
#part raid.5 --noformat --onpart sda1
#part raid.4 --noformat --onpart hdb2
#part raid.2 --noformat --onpart hda2
#raid /boot --useexisting --fstype ext2 --level=RAID1 --device=md0 raid.1 raid.3
#raid pv.13 --noformat --useexisting --fstype "physical volume (LVM)" --level=RAID1 --device=md1 raid.2 raid.4
#raid /storage --useexisting --fstype ext3 --level=RAID10 --device=md2 raid.5 raid.6 raid.7 raid.8 raid.9 raid.10
#volgroup root_lvg --noformat --useexisting --pesize=32768 pv.13
#logvol /home --useexisting --fstype ext3 --name=home_lv --vgname=root_lvg --size=1024
#logvol /var --useexisting --fstype ext3 --name=var_lv --vgname=root_lvg --size=4096
#logvol / --useexisting --fstype ext3 --name=root_lv --vgname=root_lvg --size=65952
#logvol swap --useexisting --fstype swap --name=swap_lv --vgname=root_lvg --size=4096
#logvol /tmp --useexisting --fstype ext3 --name=tmp_lv --vgname=root_lvg --size=1024
             
%packages
@base
@core
device-mapper-multipath 
vim-enhanced 
gcc
emacs-nox
screen 
-bluez-utils
-cpuspeed
-NetworkManager

Yum Configuration

Extra Repositories

Added RPMForge respository from Dag Wieers: 

rpm -Uhv http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS//rpmforge-release-0.3.6-1.el5.rf.x86_64.rpm

Added Fedora EPEL Repository from Mirror. 

rpm -Uvh http://mirror.clarkson.edu/epel//5/x86_64/epel-release-5-4.noarch.rpm

Repositories

All .repo files in /etc/yum.repos.d/ must be made to point to Mirror wherever possible. The exception is Dag Wieers, which we do not mirror. In order to accomplish this, the following files were edited: 

CentOS-Base.repo
epel.repo
epel-testing.repo
rpmforge.repo

The following files were not modified, but will appear in this documentation for the sake of completion: 

CentOS-Media.repo
mirrors-rpmforge

CentOS-Base.repo

# CentOS-Base.repo
#
# The mirror system uses the connecting IP address of the client and the
# update status of each mirror to pick mirrors that are updated to and
# geographically close to the client.  You should use this for CentOS updates
# unless you are manually picking other mirrors.
#
# If the mirrorlist= does not work for you, as a fall back you can try the 
# remarked out baseurl= line instead.
#
#

[base]
name=CentOS-$releasever - Base
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os
#baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/
baseurl=http://mirror.clarkson.edu/centos/$releasever/os/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5

#released updates 
[updates]
name=CentOS-$releasever - Updates
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates
#baseurl=http://mirror.centos.org/centos/$releasever/updates/$basearch/
baseurl=http://mirror.clarkson.edu/centos/$releasever/updates/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5

#packages used/produced in the build but not released
[addons]
name=CentOS-$releasever - Addons
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=addons
#baseurl=http://mirror.centos.org/centos/$releasever/addons/$basearch/
baseurl=http://mirror.clarkson.edu/centos/$releasever/addons/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5

#additional packages that may be useful
[extras]
name=CentOS-$releasever - Extras
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras
#baseurl=http://mirror.centos.org/centos/$releasever/extras/$basearch/
baseurl=http://mirror.clarkson.edu/centos/$releasever/extras/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5

#additional packages that extend functionality of existing packages
[centosplus]
name=CentOS-$releasever - Plus
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=centosplus
#baseurl=http://mirror.centos.org/centos/$releasever/centosplus/$basearch/
baseurl=http://mirror.clarkson.edu/centos/$releasever/centosplus/$basearch/
gpgcheck=1
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5

#contrib - packages by Centos Users
[contrib]
name=CentOS-$releasever - Contrib
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=contrib
#baseurl=http://mirror.centos.org/centos/$releasever/contrib/$basearch/
baseurl=http://mirror.clarkson.edu/centos/$releasever/contrib/$basearch/
gpgcheck=1
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5

Centos-Media.repo

Note that this repository is not enabled. 

# CentOS-Media.repo
#
# This repo is used to mount the default locations for a CDROM / DVD on
#  CentOS-5.  You can use this repo and yum to install items directly off the
#  DVD ISO that we release.
#
# To use this repo, put in your DVD and use it with the other repos too:
#  yum --enablerepo=c5-media [command]
#  
# or for ONLY the media repo, do this:
#
#  yum --disablerepo=\* --enablerepo=c5-media [command]

[c5-media]
name=CentOS-$releasever - Media
baseurl=file:///media/CentOS/
        file:///media/cdrom/
        file:///media/cdrecorder/
gpgcheck=1
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5

epel.repo

[epel]
name=Extra Packages for Enterprise Linux 5 - $basearch
#baseurl=http://download.fedoraproject.org/pub/epel/5/$basearch
#mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=epel-5&arch=$basearch
baseurl=http://mirror.clarkson.edu/epel/5/$basearch
failovermethod=priority
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL

[epel-debuginfo]
name=Extra Packages for Enterprise Linux 5 - $basearch - Debug
#baseurl=http://download.fedoraproject.org/pub/epel/5/$basearch/debug
#mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=epel-debug-5&arch=$basearch
baseurl=http://mirror.clarkson.edu/epel/5/$basearch
failovermethod=priority
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL
gpgcheck=1

[epel-source]
name=Extra Packages for Enterprise Linux 5 - $basearch - Source
#baseurl=http://download.fedoraproject.org/pub/epel/5/SRPMS
#mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=epel-source-5&arch=$basearch
baseurl=http://mirror.clarkson.edu/epel/5/$basearch
failovermethod=priority
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL
gpgcheck=1

epel-testing.repo

[epel-testing]
name=Extra Packages for Enterprise Linux 5 - Testing - $basearch
#baseurl=http://download.fedoraproject.org/pub/epel/testing/5/$basearch
#mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=testing-epel5&arch=$basearch
baseurl=http://mirror.clarkson.edu/epel/testing/5/$basearch
failovermethod=priority
enabled=0
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL

[epel-testing-debuginfo]
name=Extra Packages for Enterprise Linux 5 - Testing - $basearch - Debug
#baseurl=http://download.fedoraproject.org/pub/epel/testing/5/$basearch/debug
#mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=testing-debug-epel5&arch=$basearch
baseurl=http://mirror.clarkson.edu/epel/testing/5/$basearch
failovermethod=priority
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL
gpgcheck=1

[epel-testing-source]
name=Extra Packages for Enterprise Linux 5 - Testing - $basearch - Source
#baseurl=http://download.fedoraproject.org/pub/epel/testing/5/SRPMS
#mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=testing-source-epel5&arch=$basearch
baseurl=http://mirror.clarkson.edu/epel/testing/5/$basearch
failovermethod=priority
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL
gpgcheck=1

mirrors.rpmforge

http://apt.sw.be/redhat/el5/en/$ARCH/dag
http://archive.cs.uu.nl/mirror/dag.wieers/redhat/el5/en/$ARCH/dag
http://ftp2.lcpe.uni-sofia.bg/freshrpms/pub/dag/redhat/el5/en/$ARCH/dag
#http://ftp.heanet.ie/pub/freshrpms/pub/dag/redhat/el5/en/$ARCH/dag
http://ftp-stud.fht-esslingen.de/dag/redhat/el5/en/$ARCH/dag
http://mirror.cpsc.ucalgary.ca/mirror/dag/redhat/el5/en/$ARCH/dag
http://mirrors.ircam.fr/pub/dag/redhat/el5/en/$ARCH/dag
http://rh-mirror.linux.iastate.edu/pub/dag/redhat/el5/en/$ARCH/dag
http://rpmfind.net/linux/dag/redhat/el5/en/$ARCH/dag
http://wftp.tu-chemnitz.de/pub/linux/dag/redhat/el5/en/$ARCH/dag
http://www.mirrorservice.org/sites/apt.sw.be/redhat/el5/en/$ARCH/dag

rpmforge.repo

# Name: RPMforge RPM Repository for Red Hat Enterprise 5 - dag
# URL: http://rpmforge.net/
[rpmforge]
name = Red Hat Enterprise $releasever - RPMforge.net - dag
#baseurl = http://apt.sw.be/redhat/el5/en/$basearch/dag
#mirrorlist = http://apt.sw.be/redhat/el5/en/mirrors-rpmforge
#mirrorlist = file:///etc/yum.repos.d/mirrors-rpmforge
baseurl = http://mirror.clarkson.edu/rpmforge/redhat/el5/en/$basearch/dag
enabled = 1
protect = 0
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rpmforge-dag
gpgcheck = 1


Final Yum Config

Disable Yum fastestmirror plugin since we are pulling from Mirror only. 

sed -i 's/enabled=1/enabled=0/g' /etc/yum/pluginconf.d/fastestmirror.conf

Install yum priorities plugin: 

yum install yum-priorities

Configure yum priorities to check for obsoletes: 

echo "check_obsoletes=1" >> /etc/yum/pluginconf.d/priorities.conf

Install a few useful extra packages: 

yum install vim-enhanced gcc emacs-nox screen iftop

Update the system: 

yum update

User Configuration

Adding Users

Created users for all maintainers / interested parties (don't forget to set passwords with passwd <username>): 

useradd -m platekme
useradd -m kopptr
useradd -m mccarrms

Add administrative users to group 'wheel' so they can use sudo. For example, 

gpasswd -a platekme wheel

Configure Sudo

Edit /etc/sudoers the safe way with the command 

visudo

Note that this configuration allows all users in group 'wheel' to use sudo. 

## Sudoers allows particular users to run various commands as
## the root user, without needing the root password.
##
## Examples are provided at the bottom of the file for collections
## of related commands, which can then be delegated out to particular
## users or groups.
## 
## This file must be edited with the 'visudo' command.

## Host Aliases
## Groups of machines. You may prefer to use hostnames (perhap using 
## wildcards for entire domains) or IP addresses instead.
# Host_Alias     FILESERVERS = fs1, fs2
# Host_Alias     MAILSERVERS = smtp, smtp2

## User Aliases
## These aren't often necessary, as you can use regular groups
## (ie, from files, LDAP, NIS, etc) in this file - just use %groupname 
## rather than USERALIAS
# User_Alias ADMINS = jsmith, mikem


## Command Aliases
## These are groups of related commands...

## Networking
Cmnd_Alias NETWORKING = /sbin/route, /sbin/ifconfig, /bin/ping, /sbin/dhclient, /usr/bin/net, /sbin/iptables, /usr/bin/rfcomm, /usr/bin/wvdial, /sbin/iwconfig, /sbin/mii-tool

## Installation and management of software
Cmnd_Alias SOFTWARE = /bin/rpm, /usr/bin/up2date, /usr/bin/yum

## Services
Cmnd_Alias SERVICES = /sbin/service, /sbin/chkconfig

## Updating the locate database
Cmnd_Alias LOCATE = /usr/bin/updatedb

## Storage
Cmnd_Alias STORAGE = /sbin/fdisk, /sbin/sfdisk, /sbin/parted, /sbin/partprobe, /bin/mount, /bin/umount

## Delegating permissions
Cmnd_Alias DELEGATING = /usr/sbin/visudo, /bin/chown, /bin/chmod, /bin/chgrp

## Processes
Cmnd_Alias PROCESSES = /bin/nice, /bin/kill, /usr/bin/kill, /usr/bin/killall

## Drivers
Cmnd_Alias DRIVERS = /sbin/modprobe

## Shells
Cmnd_Alias SHELLS = /bin/sh, /bin/bash, /usr/bin/rsh, /bin/dash, /bin/rbash, /bin/su

## Users
Cmnd_Alias USERS = /usr/sbin/useradd, /usr/sbin/userdel, /usr/sbin/userhelper, /usr/sbin/usermod, /usr/sbin/usernetctl

# Defaults specification

#
# Disable "ssh hostname sudo <cmd>", because it will show the password in clear. 
#         You have to run "ssh -t hostname sudo <cmd>".
#
Defaults    requiretty

Defaults    env_reset,tty_tickets,lecture=always,logfile=/var/log/sudo.log
Defaults    env_keep = "COLORS DISPLAY HOSTNAME HISTSIZE INPUTRC KDEDIR \
                        LS_COLORS MAIL PS1 PS2 QTDIR USERNAME \
                        LANG LC_ADDRESS LC_CTYPE LC_COLLATE LC_IDENTIFICATION \
                        LC_MEASUREMENT LC_MESSAGES LC_MONETARY LC_NAME LC_NUMERIC \
                        LC_PAPER LC_TELEPHONE LC_TIME LC_ALL LANGUAGE LINGUAS \
                        _XKB_CHARSET XAUTHORITY"

## Next comes the main part: which users can run what software on 
## which machines (the sudoers file can be shared between multiple
## systems).
## Syntax:
##
##      user    MACHINE=COMMANDS
##
## The COMMANDS section may have other options added to it.
##
## Allow root to run any commands anywhere 
root    ALL=(ALL)       ALL

## Allows members of the 'sys' group to run networking, software, 
## service management apps and more.
# %sys ALL = NETWORKING, SOFTWARE, SERVICES, STORAGE, DELEGATING, PROCESSES, LOCATE, DRIVERS

## Allows people in group wheel to run all commands
# %wheel        ALL=(ALL)       ALL

## Same thing without a password
%wheel  ALL=(ALL)       NOPASSWD: ALL

## Allows members of the users group to mount and unmount the 
## cdrom as root
# %users  ALL=/sbin/mount /mnt/cdrom, /sbin/umount /mnt/cdrom

## Allows members of the users group to shutdown this system
# %users  localhost=/sbin/shutdown -h now

Change System PATH Variable

Edit /etc/profile: 

PATH=$PATH:/usr/sbin:/sbin
export PATH

Modify Root's Crontab

Add the following entry to root's crontab with the command crontab -e 

# Used to update locate database
0 * * * * /usr/bin/updatedb

Disable CTRL-ALT-DELETE

Remove trap entry to prevent accidental reboots, then make changes active: 

sed -i 's/ca::ctrlaltdel:/#ca::ctrlaltdel:/g' /etc/inittab
init q


Network Configuration

Hostname

Configure the hostname in /etc/sysconfig/network: 

NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=storage
GATEWAY=128.153.145.1

Interfaces

Verified eth0 configuration for Clarkson Network in /etc/sysconfig/network-scripts/ifcfg-eth0 

# Intel Corporation 82541PI Gigabit Ethernet Controller
DEVICE=eth0
BOOTPROTO=static
BROADCAST=128.153.145.255
HWADDR=00:1B:21:28:C8:6A
IPADDR=128.153.145.40
NETMASK=255.255.255.0
NETWORK=128.153.145.0
ONBOOT=yes

Verified eth1 configuration for the Server Room Network in /etc/sysconfig/network-scripts/ifcfg-eth1 

# Intel Corporation 82566DM-2 Gigabit Network Connection
DEVICE=eth1
BOOTPROTO=static
BROADCAST=10.0.1.255
HWADDR=00:30:48:9A:DA:5E
IPADDR=10.0.1.35
NETMASK=255.255.255.0
NETWORK=10.0.1.0
ONBOOT=yes

Verified eth2 configuration for the COSI Internal Network in /etc/sysconfig/network-scripts/ifcfg-eth2 

# Intel Corporation 82573L Gigabit Ethernet Controller
DEVICE=eth2
BOOTPROTO=static
BROADCAST=10.0.0.255
HWADDR=00:30:48:9A:DA:5F
IPADDR=10.0.0.15
NETMASK=255.255.255.0
NETWORK=10.0.0.0
ONBOOT=yes

Hosts

Edited /etc/hosts: 

# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1       localhost.localdomain localhost
128.153.145.40  storage.clarkson.edu storage.cslabs.clarkson.edu storage.cslabs storage
10.0.1.35       storage.sr.cslabs.clarkson.edu storage.sr.cslabs storage.sr
10.0.0.15       storage.int.cslabs.clarkson.edu storage.int.cslabs storage.int

Edited /etc/hosts.allow: 

This file has intentionally been left out for security reasons.

Edited /etc/hosts.deny: 

ALL: ALL

DNS

Edited /etc/resolv.conf: 

search cslabs.clarkson.edu clarkson.edu
nameserver 128.153.145.3
nameserver 128.153.145.4

IP Configuration

Disable IPv6

IPv6 should be disabled since we don't use it. First, add the following to /etc/modprobe.conf: 

install ipv6 /bin/true

Then, disable the unneeded IPv6 firewall with this command: 

/sbin/chkconfig ip6tables off

IPtables Configuration

This configuration file has been omitted for security reasons.

After configuring it, restart iptables: 

/etc/init.d/iptables restart

SSH Configuration

Edited file /etc/ssh/sshd_config: 

This configuration file has been omitted for security concerns.

After editing the config file, restart sshd: 

/etc/init.d/sshd restart

Use the ASCII Art text generator at http://patorjk.com/software/taag/ to generate the login banner in font 'small slant'. Put this login banner in /etc/issue.net: 

   ______                        
  / __/ /____  _______ ____ ____ 
 _\ \/ __/ _ \/ __/ _ `/ _ `/ -_)
/___/\__/\___/_/  \_,_/\_, /\__/ 
                      /___/

Set user password requirements by editing /etc/login.defs: 

MAIL_DIR        /var/spool/mail

PASS_MAX_DAYS   360
PASS_MIN_DAYS   0
PASS_MIN_LEN    8
PASS_WARN_AGE   60

UID_MIN                   500
UID_MAX                 60000

GID_MIN                   500
GID_MAX                 60000

CREATE_HOME     yes

UMASK           077

USERGROUPS_ENAB yes

MD5_CRYPT_ENAB yes

ENCRYPT_METHOD MD5

NTP

First, install NTP: 

yum install ntp

Edit /etc/ntp.conf: 

restrict default kod nomodify notrap nopeer noquery
restrict -6 default kod nomodify notrap nopeer noquery

restrict 127.0.0.1
restrict -6 ::1

restrict tick.clarkson.edu mask 255.255.255.255 nomodify notrap noquery
restrict tock.clarkson.edu mask 255.255.255.255 nomodify notrap noquery

server tick.clarkson.edu
server tock.clarkson.edu

server  127.127.1.0     # local clock
fudge   127.127.1.0 stratum 10

driftfile /var/lib/ntp/drift

keys /etc/ntp/keys

Edit /etc/ntp/step-tickers: 

tick.clarkson.edu
tock.clarkson.edu

Configure NTP to start on boot, and start it now: 

chkconfig --levels 2345 ntpd on
/etc/init.d/ntpd start

Configure NTP to sync the hardware clock. Edit /etc/sysconfig/ntpd: 

# Drop root to id 'ntp:ntp' by default.
OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid"

# Set to 'yes' to sync hw clock after successful ntpdate
SYNC_HWCLOCK=yes

# Additional options for ntpdate
NTPDATE_OPTIONS=""

Power Management

APCUPSD

This package is used to monitor the UPS which Storage is plugged into and is used to shutdown the system in the event of a power failure.

Change the BIOS settings to cause the machine to be powered on when power is restored. This will be something like Restore on AC/Power Loss.

Install APCUPSD 

yum install apcupsd

Edit /etc/apcupsd/apcupsd.conf: 

## apcupsd.conf v1.1 ##

UPSNAME ups3

UPSCABLE ether

UPSTYPE net
DEVICE 128.153.145.215:3551

LOCKFILE /var/lock

SCRIPTDIR /etc/apcupsd

PWRFAILDIR /etc/apcupsd

NOLOGINDIR /etc

ONBATTERYDELAY 6

BATTERYLEVEL 10

MINUTES 15

TIMEOUT 0

ANNOY 300

ANNOYDELAY 60

NOLOGON disable

KILLDELAY 0

NETSERVER on

NISIP 127.0.0.1

NISPORT 3551

EVENTSFILE /var/log/apcupsd.events

EVENTSFILEMAX 10

UPSCLASS standalone

UPSMODE disable

STATTIME 0

STATFILE /var/log/apcupsd.status

LOGSTATS off

DATATIME 0

SELFTEST 336

Configure apcupsd to start on boot, and start it: 

/sbin/chkconfig --levels 2345 apcupsd on
/etc/init.d/apcupsd start

Mail and Logging

Configure Aliases

Edit /etc/aliases: 

#
#  Aliases in this file will NOT be expanded in the header from
#  Mail, but WILL be visible over networks or from /bin/mail.
#
#       >>>>>>>>>>      The program "newaliases" must be run after
#       >> NOTE >>      this file is updated for any changes to
#       >>>>>>>>>>      show through to sendmail.
#

# Basic system aliases -- these MUST be present.
mailer-daemon:  postmaster
postmaster:     logwatch@cslabs.clarkson.edu

# General redirections for pseudo accounts.
bin:            logwatch@cslabs.clarkson.edu
daemon:         logwatch@cslabs.clarkson.edu
adm:            logwatch@cslabs.clarkson.edu
lp:             logwatch@cslabs.clarkson.edu
sync:           logwatch@cslabs.clarkson.edu
shutdown:       logwatch@cslabs.clarkson.edu
halt:           logwatch@cslabs.clarkson.edu
mail:           logwatch@cslabs.clarkson.edu
news:           logwatch@cslabs.clarkson.edu
uucp:           logwatch@cslabs.clarkson.edu
operator:       logwatch@cslabs.clarkson.edu
games:          logwatch@cslabs.clarkson.edu
gopher:         logwatch@cslabs.clarkson.edu
ftp:            logwatch@cslabs.clarkson.edu
nobody:         logwatch@cslabs.clarkson.edu
radiusd:        logwatch@cslabs.clarkson.edu
nut:            logwatch@cslabs.clarkson.edu
dbus:           logwatch@cslabs.clarkson.edu
vcsa:           logwatch@cslabs.clarkson.edu
canna:          logwatch@cslabs.clarkson.edu
wnn:            logwatch@cslabs.clarkson.edu
rpm:            logwatch@cslabs.clarkson.edu
nscd:           logwatch@cslabs.clarkson.edu
pcap:           logwatch@cslabs.clarkson.edu
apache:         logwatch@cslabs.clarkson.edu
webalizer:      logwatch@cslabs.clarkson.edu
dovecot:        logwatch@cslabs.clarkson.edu
fax:            logwatch@cslabs.clarkson.edu
quagga:         logwatch@cslabs.clarkson.edu
radvd:          logwatch@cslabs.clarkson.edu
pvm:            logwatch@cslabs.clarkson.edu
amanda:         logwatch@cslabs.clarkson.edu
privoxy:        logwatch@cslabs.clarkson.edu
ident:          logwatch@cslabs.clarkson.edu
named:          logwatch@cslabs.clarkson.edu
xfs:            logwatch@cslabs.clarkson.edu
gdm:            logwatch@cslabs.clarkson.edu
mailnull:       logwatch@cslabs.clarkson.edu
postgres:       logwatch@cslabs.clarkson.edu
sshd:           logwatch@cslabs.clarkson.edu
smmsp:          logwatch@cslabs.clarkson.edu
postfix:        logwatch@cslabs.clarkson.edu
netdump:        logwatch@cslabs.clarkson.edu
ldap:           logwatch@cslabs.clarkson.edu
squid:          logwatch@cslabs.clarkson.edu
ntp:            logwatch@cslabs.clarkson.edu
mysql:          logwatch@cslabs.clarkson.edu
desktop:        logwatch@cslabs.clarkson.edu
rpcuser:        logwatch@cslabs.clarkson.edu
rpc:            logwatch@cslabs.clarkson.edu
nfsnobody:      logwatch@cslabs.clarkson.edu

ingres:         logwatch@cslabs.clarkson.edu
system:         logwatch@cslabs.clarkson.edu
toor:           logwatch@cslabs.clarkson.edu
manager:        logwatch@cslabs.clarkson.edu
dumper:         logwatch@cslabs.clarkson.edu
abuse:          logwatch@cslabs.clarkson.edu

newsadm:        news
newsadmin:      news
usenet:         news
ftpadm:         ftp
ftpadmin:       ftp
ftp-adm:        ftp
ftp-admin:      ftp
www:            webmaster
webmaster:      logwatch@cslabs.clarkson.edu
noc:            logwatch@cslabs.clarkson.edu
security:       logwatch@cslabs.clarkson.edu
hostmaster:     logwatch@cslabs.clarkson.edu
info:           postmaster
marketing:      postmaster
sales:          postmaster
support:        postmaster


# trap decode to catch security attacks
decode:         logwatch@cslabs.clarkson.edu

# Person who should get roots's mail
root:           logwatch@cslabs.clarkson.edu

With that config file in place, update the aliases: 

/usr/bin/newaliases












Disabled Various Kernel Modules

  • Added the following to /etc/modprobe.conf
install pppox /bin/true
install bluetooth /bin/true
install sctp /bin/true

Installed & Configured SNMP

  • Installed needed packages
yum install net-snmp ntp
  • Configured SNMP Daemon /etc/snmp/snmpd.conf
rocommunity     <passphrase>  127.0.0.1
rocommunity     <passphrase>  <ipsallowed>
 
syslocation Clarkson University Applied CS Labs
syscontact Matt McCarrell <mccarrms@gmail.com>
disk /
disk /var
disk /boot
disk /mnt/raid
disk /mnt/lvg_storage
disk /mnt/storage1
disk /mnt/storage2
exec timeskew /usr/local/sbin/ntp_check
exec uptime /usr/bin/uptime
  • Deployed ntp_check script
    • Copied over /usr/local/sbin/ntp_check from Isengard to /usr/local/sbin/
    • chown root.root /usr/local/sbin/ntp_check
  • Configured SNMP to start at specific run levels
/sbin/chkconfig --levels 2345 snmpd on
  • Started daemon
/etc/init.d/snmpd start

Increased Detail of Logwatch Reports

  • Set detail level to be high
echo "Detail = High" >> /etc/logwatch/conf/logwatch.conf

Disabled Unneeded Services

chkconfig nfs off
/etc/init.d/nfs stop
chkconfig nfslock off
/etc/init.d/nfslock stop
chkconfig rpcgssd off
/etc/init.d/rpcgssd stop
chkconfig rpcidmapd off
/etc/init.d/rpcidmapd stop
chkconfig rpcsvcgssd off
/etc/init.d/rpcsvcgssd stop
chkconfig portmap off
/etc/init.d/portmap stop
chkconfig netfs off
/etc/init.d/netfs stop
chkconfig anacron off
/etc/init.d/anacron stop
chkconfig autofs off
/etc/init.d/autofs stop
chkconfig avahi-daemon off
/etc/init.d/avahi-daemon stop
chkconfig avahi-dnsconfd off
/etc/init.d/avahi-dnsconfd stop
chkconfig bluetooth off
/etc/init.d/bluetooth stop
chkconfig hidd off
/etc/init.d/hidd stop
chkconfig cups off
/etc/init.d/cups stop
chkconfig firstboot off
/etc/init.d/firstboot stop
chkconfig gpm off
/etc/init.d/gpm stop
chkconfig haldaemon off
/etc/init.d/haldaemon stop
chkconfig irda off
/etc/init.d/irda stop
chkconfig kudzu off
/etc/init.d/kudzu stop
chkconfig messagebus off
/etc/init.d/messagebus stop
chkconfig microcode_ctl off
/etc/init.d/microcode_ctl stop
chkconfig pcscd off
/etc/init.d/pcscd stop
chkconfig readahead_early off
/etc/init.d/readahead_early stop
chkconfig readahead_later off
/etc/init.d/readahead_later stop
chkconfig ypbind off
/etc/init.d/ypbind stop

Modified /etc/fstab to increase performance

  • Configured to not update inode access times
/dev/root_lvg/root_lv   /                       ext3    defaults        1 1
/dev/md2                /mnt/raid               ext3    defaults,noatime        1 2
/dev/storage_lvg/storage_lv /mnt/lvg_storage        ext3    defaults,noatime        1 2
/dev/md0                /boot                   ext3    defaults        1 2
/dev/root_lvg/var_lv    /var                    ext3    defaults        1 2
LABEL=/mnt/storage1     /mnt/storage1           ext3    defaults,noatime        1 2
LABEL=/mnt/storage2     /mnt/storage2           ext3    defaults,noatime        1 2
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
/dev/root_lvg/swap_lv   swap                    swap    defaults        0 0
Retrieved from "http://docs.cslabs.clarkson.edu/mediawiki/index.php?title=Storage_Setup_Process&oldid=4889"