Difference between revisions of "Mirror Setup Process"

From CSLabsWiki
Jump to: navigation, search
m (fixed some formatting)
(Updated)
Line 4: Line 4:
  
 
=Install=
 
=Install=
See [[Centos Server Setup]]
+
See [[Debian Server Setup]]
  
==CentOS 6.0 x64==
+
==Debian 7 x64==
*69 GB  / - Software RAID 1
+
*71 GB  / - Software RAID 1
*1008 MB /boot - Software RAID 1
+
*4GB Swap per OS Drive
*4GB Swap
 
  
 
=Setup=
 
=Setup=
Line 15: Line 14:
  
 
==Raid==
 
==Raid==
*/dev/sd[abcdefg]
+
*sda sdb sdc sde sdg sdh
*/dev/md0
+
*/dev/md2
*Software RAID 6
+
*Software RAID 10
*/storage
+
*/storage xfs
  
 
==Networking==
 
==Networking==
  
===eth0===
+
===eth2===
 
*Intel Corporation 82541PI Gigabit Ethernet Controller
 
*Intel Corporation 82541PI Gigabit Ethernet Controller
 
*HWADDR=00:1B:21:28:C8:48
 
*HWADDR=00:1B:21:28:C8:48
Line 32: Line 31:
 
*IPADDR=10.0.1.36
 
*IPADDR=10.0.1.36
  
===eth2===
+
===eth3===
 
*Intel Corporation 82573L Gigabit Ethernet Controller
 
*Intel Corporation 82573L Gigabit Ethernet Controller
 
*HWADDR=00:30:48:9A:DB:27
 
*HWADDR=00:30:48:9A:DB:27
Line 46: Line 45:
 
                            
 
                            
 
</pre></code>
 
</pre></code>
 +
==SSHD config==
 +
uncomment
 +
Banner /etc/issue.net
  
===Added Custom PATH Variables===
+
==Installed nginx==
*Added the following to /etc/profile
+
  apt-get install install nginx
  PATH=$PATH:/usr/sbin:/sbin
 
export PATH
 
<!--
 
===Installed and Configured [http://www.apcupsd.org/ APCUPSD]===
 
This package is used to monitor the UPS which [[Mirror]] is plugged into and is used to shutdown the system in the event of a power failure.
 
 
 
====Configured to Power On when Power is Restored====
 
*Edited the BIOS to have <code>Restore on AC/Power Loss</code> set to <code>Power On</code>.
 
 
 
====Installed and configured <code>apcupsd</code>====
 
*Installed <code>apcupsd</code>
 
**<code>yum install apcupsd</code>
 
 
 
*Edited <code>/etc/apcupsd/apcupsd.conf</code>
 
<code><pre>
 
## apcupsd.conf v1.1 ##
 
 
 
UPSNAME ups3
 
 
 
UPSCABLE ether
 
 
 
UPSTYPE net
 
DEVICE 128.153.145.215:3551
 
 
 
LOCKFILE /var/lock
 
 
 
SCRIPTDIR /etc/apcupsd
 
 
 
PWRFAILDIR /etc/apcupsd
 
 
 
NOLOGINDIR /etc
 
 
 
ONBATTERYDELAY 6
 
 
 
BATTERYLEVEL 10
 
 
 
MINUTES 15
 
 
 
TIMEOUT 0
 
 
 
ANNOY 300
 
 
 
ANNOYDELAY 60
 
 
 
NOLOGON disable
 
 
 
KILLDELAY 0
 
 
 
NETSERVER on
 
 
 
NISIP 127.0.0.1
 
 
 
NISPORT 3551
 
 
 
EVENTSFILE /var/log/apcupsd.events
 
 
 
EVENTSFILEMAX 10
 
 
 
UPSCLASS standalone
 
 
 
UPSMODE disable
 
 
 
STATTIME 0
 
 
 
STATFILE /var/log/apcupsd.status
 
 
 
LOGSTATS off
 
 
 
DATATIME 0
 
 
 
SELFTEST 336
 
</pre></code>
 
 
 
*Configured <code>apcupsd</code> to start on boot
 
**<code>/sbin/chkconfig --levels 2345 apcupsd on</code>
 
 
 
*Started <code>apcupsd</code>
 
**<code>/etc/init.d/apcupsd start</code>
 
-->
 
 
 
==Installed Apache==
 
yum install httpd
 
 
 
*Modified /etc/httpd/conf/httpd.conf
 
<code><pre>
 
(Replace)
 
</pre></code>
 
 
 
*Configured Apache to start on boot
 
/sbin/chkconfig --levels 345 httpd on
 
 
 
*Started Apache
 
service httpd start
 
 
 
====Modified logrotate settings====
 
/etc/logrotate.d/httpd
 
<code><pre>
 
/var/log/httpd/*log {
 
    weekly
 
    rotate 1
 
    missingok
 
    notifempty
 
    sharedscripts
 
    postrotate
 
        /sbin/service httpd reload > /dev/null 2>/dev/null || true
 
    endscript
 
}
 
</pre></code>
 
 
 
====AWStats====
 
*Mirror is configured to be monitored by AWStats on [[Netstat]].  The basic process for doing this is on the [[Configure a Remote Red Hat Based Linux Server for AWStats]] how-to.
 
  
 
==Configure rsync to run as a daemon==
 
==Configure rsync to run as a daemon==
 
*Installed <code>xinetd</code>
 
*Installed <code>xinetd</code>
  yum install xinetd
+
  apt-get install rsync
  
*Enabled rsync /etc/xinetd.d/rsync
+
*Configured rsync /etc/rsyncd.conf
<code><pre>
 
# default: off
 
# description: The rsync server is a good addition to an ftp server, as it #  allows crc checksumming etc.
 
service rsync
 
{
 
    disable = no
 
    socket_type    = stream
 
    wait            = no
 
    user            = root
 
    server          = /usr/bin/rsync
 
    server_args    = --daemon
 
    log_on_failure  += USERID
 
}
 
</pre></code>
 
 
 
*Created configuration and shares for rsync daemon /etc/rsyncd.conf
 
 
<code><pre>
 
<code><pre>
 
uid = nobody
 
uid = nobody
gid = nobody
+
gid = nogroup
 
use chroot = yes
 
use chroot = yes
 
max connections = 20
 
max connections = 20
Line 202: Line 77:
 
   exclude = lost+found/
 
   exclude = lost+found/
 
   read only = true
 
   read only = true
   ignore nonreadable = yes
+
   ignore nonreadable = yes</pre></code>
...
 
</pre></code>
 
  
 
*Created rsync motd /etc/rsyncd.motd
 
*Created rsync motd /etc/rsyncd.motd
Line 223: Line 96:
 
###############################################################
 
###############################################################
 
</pre></code>
 
</pre></code>
 
*Secured rsync
 
chown root.root /etc/rsyncd.*
 
 
chmod 600 /etc/rsyncd.*
 
 
*Restarted xinetd
 
servic xinetd restart
 
  
 
==Set Up Fedora Mirror Manager==
 
==Set Up Fedora Mirror Manager==
 
*Installed mirrormanager
 
*Installed mirrormanager
  yum install mirrormanager-client
+
  cd /usr/local/sbin
 +
git clone git://git.fedorahosted.org/mirrormanager/
 +
ln -s /usr/local/sbin/mirrormanager/client/report_mirror /usr/local/sbin/report_mirror
 +
  
 
*Configured mirrormanager
 
*Configured mirrormanager
Line 327: Line 195:
 
enabled=0
 
enabled=0
 
path=/var/www/html/pub/fedora-archive
 
path=/var/www/html/pub/fedora-archive
 +
</pre></code>
 +
 +
*Added cron entries for root
 +
<code><pre>
 +
# Dir sizes
 +
0 0 * * * /usr/local/bin/dir_sizes.sh
 +
 +
# backup
 +
45 0 * * * /bin/nice -n 19 /usr/bin/ionice -c2 -n7 /usr/local/bin/backup.sh
 +
 +
10 0 * * * /usr/local/sbin/report_mirror
 
</pre></code>
 
</pre></code>

Revision as of 14:19, 19 April 2014


This page summarizes how Mirror was set up in Fall 2011.

Install

See Debian Server Setup

Debian 7 x64

  • 71 GB / - Software RAID 1
  • 4GB Swap per OS Drive

Setup

See Centos Server Setup

Raid

  • sda sdb sdc sde sdg sdh
  • /dev/md2
  • Software RAID 10
  • /storage xfs

Networking

eth2

  • Intel Corporation 82541PI Gigabit Ethernet Controller
  • HWADDR=00:1B:21:28:C8:48
  • IPADDR=128.153.145.19

eth1

  • Intel Corporation 82566DM-2 Gigabit Network Connection
  • HWADDR=00:30:48:9A:DB:26
  • IPADDR=10.0.1.36

eth3

  • Intel Corporation 82573L Gigabit Ethernet Controller
  • HWADDR=00:30:48:9A:DB:27
  • IPADDR=10.0.0.14

Set Up SSH Login Banner

/etc/issue.net

         _                 
  __ _  (_)__________  ____
 /  ' \/ / __/ __/ _ \/ __/
/_/_/_/_/_/ /_/  \___/_/   
                           

SSHD config

uncomment

Banner /etc/issue.net

Installed nginx

apt-get install install nginx

Configure rsync to run as a daemon

  • Installed xinetd
apt-get install rsync
  • Configured rsync /etc/rsyncd.conf
uid = nobody
gid = nogroup
use chroot = yes
max connections = 20
pid file = /var/run/rsyncd.pid
motd file = /etc/rsyncd.motd
log file = /var/log/rsync.log
transfer logging = yes
log format = %t %a %m %f %b
syslog facility = local3
timeout = 900
dont compress = *.gz *.tgz *.zip *.z *.Z *.rpm *.deb *.bz2
refuse options = checksum

[name]
  comment = Name of Repository
  path = /storage/repository
  exclude = lost+found/
  read only = true
  ignore nonreadable = yes
  • Created rsync motd /etc/rsyncd.motd
###############################################################

The Clarkson University mirror is located in Potsdam, NY, US.

This mirror is operated by the Clarkson Open Source Institute.
http://cosi.clarkson.edu/

If you have any questions or problems, please email
mirror-admin@cslabs.clarkson.edu.

Recent changes can be found at
http://status.cslabs.clarkson.edu/tag/mirror.

###############################################################

Set Up Fedora Mirror Manager

  • Installed mirrormanager
cd /usr/local/sbin
git clone git://git.fedorahosted.org/mirrormanager/
ln -s /usr/local/sbin/mirrormanager/client/report_mirror /usr/local/sbin/report_mirror

  • Configured mirrormanager
    • Modified /etc/mirrormanager-client/report_mirror.conf
[global]
# if enabled=0, no data is sent to the database
enabled=1
# server= is the URL to the MirrorManager XML-RPC interface
server=https://admin.fedoraproject.org/mirrormanager/xmlrpc


[site]
# if enabled=0, no data about this site is sent to the database
enabled=1
# Name and Password fields need to match the Site name and password
# fields you entered for your Site in the MirrorManager database at
# https://admin.fedoraproject.org/mirrormanager
name=Clarkson University
password=<PASSWORD GOES HERE>

[host]
# if enabled=0, no data about this host is sent to the database
enabled=1
# Name field need to match the Host name field you entered for your
# Host in the MirrorManager database at
# https://admin.fedoraproject.org/mirrormanager
name=mirror.clarkson.edu
# if user_active=0, no data about this category is given to the public
# This can be used to toggle between serving and not serving data,
# such enabled during the nighttime (when you have more idle bandwidth
# available) and disabled during the daytime.
# By not specifying user_active, the database will not be updated.
# user_active=1

[stats]
# Stats are only sent when run with the -s option
# and when this section is enabled.
# This feature is not presently implemented
enabled=0
apache=/var/log/httpd/access_log
vsftpd=/var/log/vsftpd.log
# remember to enable log file and transfer logging in rsyncd.conf
rsyncd=/var/log/rsyncd.log


# Content Categories
# These sections match the Categories for content tracked by MirrorManager.
#
# enabled=1 means information about this category will be sent to the database.
# enabled=0, no data about this host is sent to the database.  If the
# database already has information for you for this Category, it will
# remain unchanged.  This can be used to update the database after you
# have manually synced some infrequently-updated content, such as
# historical releases.
#
# path= is the path on your local disk to the top-level directory for this Category

[Fedora Linux]
enabled=1
path=/var/www/html/fedora/linux

[Fedora EPEL]
enabled=1
path=/var/www/html/epel

# lesser used categories below

[Fedora Web]
enabled=0
path=/var/www/html/pub/fedora/web

[Fedora Secondary Arches]
enabled=0
path=/var/www/html/pub/fedora-secondary

[Fedora Other]
enabled=0
path=/var/www/html/pub/alt

# historical content

[Fedora Core]
enabled=0
path=/var/www/html/pub/fedora/linux/core

[Fedora Extras]
enabled=0
path=/var/www/html/pub/fedora/linux/extras

[Fedora Archive]
enabled=0
path=/var/www/html/pub/fedora-archive
  • Added cron entries for root
# Dir sizes
0 0 * * * /usr/local/bin/dir_sizes.sh

# backup
45 0 * * * /bin/nice -n 19 /usr/bin/ionice -c2 -n7 /usr/local/bin/backup.sh

10 0 * * * /usr/local/sbin/report_mirror