Network Setup

From CSLabsWiki
Jump to: navigation, search

Basics

The connection to OIT is provided by a Gigabit CAT5e cable. This CAT5e cable plugs into the OIT switch's Gigabit port 2 (sc-344-a). There is also a patch cable that connects port 46 of the 100M ports on sc344 which provides the 128.153.146.0/24. A patch cable connects the outbound Gigabit port 2 to SWM1. From SWM1, there are 2 branches to ITL switches, branches to Ziltoid (COSI's Firewall), Mirror and Tor-Exit. SWM2 and SWM3 are inside the server racks and distribute network to those racks.

Behind Ziltoid, the patch cables for all the Ethernet and the WiFi inside COSI itself are connected to a handful of switches. On a seperate two ports on Ziltoid there are cables to SWM2 and SWM3

The ITL is connected into some 24 and 48 port switches which distribute network directly. A third switch which is normally unplugged is used to give the ITL network internal connectivity for botnet and virus research.

STP

STP is enabled on all ports on the managed switches, and when a loop is detected, the port will shut down. Once a loop has been resolved, it can take up to 10 seconds before the switch tries again.

DO NOT EVER ENABLE STP ON THE OIT UPLINK PORTS!! YOU WILL GET DISCONNECTED!

Those ports are port 1 and port 19 on SWM1.

If you do this, submit an OIT RT ticket saying you're sorry, and that you need the port reset, because BPDU guard detected a STP packet. Just one is all it takes.

Don't ask them to disable it either, they will get salty (and for good reason).

DHCP/DNS/etc

DHCP, DNS, and LDAP/Kerberos are all handled by Talos

Parts of 128.153.144.0/24 are on DHCP

All of 128.153.145.0/24 are static allocation (by host)

Portmaps

This is a list of ports to which are connected to our managed switches. It is highly advised not to move machines from their location on the switches without permission from someone who knows the per-port configuration on the switch in question, otherwise some machines may not be properly throttled or managed, resulting in (dire) consequences.

Upload and Download limits are machine centric. The internal switch dialogs are switch centric, thus "ingress" from switch to machine is upload and "egress" from switch to machine is download.

SWM1 Portmap

Ports 1 and 19 should NEVER see STP or you will be sad. NEVER CHANGE UPLINK PORTS!

Port Allocation VLAN Upload Kb/s Download Kb/s
1 OIT Link (sc-344-a, port 2) 1, Access
2 DEBUG 1, Access
4 Ziltoid 1, Access
5 ITL SW6 1, Access
7 ITL SW5 1, Access
10 Mirror 1, Access 750016 299968
17 NC 2, Access
18 NC 2, Access
19 OIT Uplink (sc-334-a, port 46) 2, Access
20 Tor-Exit 2, Access

SWM2 Portmap

Port Allocation Upload Kb/s Download Kb/s
1 Ziltoid
2 DEBUGGING
3 SWM3
4 Bacon
6 Bennu
10 The-Internet
12 NAS1
14 Talos
17 Red Dwarf 99968
19 Prometheus 99968

SWM3 Portmap

Port Allocation Upload Kb/s Download Kb/s
1 Ziltoid
2 DEBUGGING
3 SWM2
6 Elephant
10 Gitlab
11 Hydra@ETH1
12 Hydra@ETH0
13 Grand-Dad@ETH0
14 Grand-Dad@ETH1
16 Androbattery
21 Management-Monitor
22 Norm

On-Switch Portmaps

All of our managed switches contain descriptions of the port; after logging in, proceed to Switching > Port, and, under Port Config (the default pane), you will see the Description column. The data in those columns should correspond to what you see above; if not, please consult the physical status of the network and update both accordingly.

The Description field stored by our TP-Link switches is only 16 characters wide and can contain only alphanumerics and the characters "@-_:/.", which significantly limits the usability of this field. Nonetheless, I've come up with the following schema for the field:

type:name@ident

The type field may contain any of the following:

  • UL indicates an uplink--a hop to a router with a smaller prefix (more general routing) or closer to the center of the hub in a hub-spoke network.
  • DL indicates a downlink--a hop to a router with a larger prefix (toward a host) or closer to the edges of a hub-spoke network.
  • XL indicates a crosslink--a hop between routers with the same prefix, often included for redundancy.
  • UL:H, DL:H, and XL:H indicate hosts--not routers or switches per se, but general-purpose computers--that are responsible for forwarding traffic as an uplink, downlink, or crosslink, respectively. Firewalls (such as Ziltoid) and host-based NATs (such as DDC-router) fall into this category.
  • H is a host--a general purpose computer. Most of our services, including repositories (Mirror), VM hosts (Bennu, Hydra), tunnelling endpoints Tor-exit, and general services (Androbattery) fall into this category. At present, this scheme only considers physical hosts, and so it does not count, e.g., VMs which are responsible for routing--this may change if there is a need for it.
  • DEBUG is an open port reserved for debugging the switch--it should always have access to the switch's management interface. Such ports generally have no name.

Not all uplinks and downlinks are reciprocated in this configuration; for example, the "downlinked" switch may not be managed, and thus not be able to have port descriptions.

The name field should contain the canonical name of the remote end of that port's cable; for switches, this is usually named "SW*" (e.g., SWM1, SW0, etc.), and for hosts, this is either its DNS name or its self-recognized hostname. If there exists no sensible name for this connection, the name may be elided (along with the colon :), but only if the ident field may also be elided. An empty name is permitted in the latter circumstance.

The ident name should be a non-negative integer that can disambiguate multiple links between the same switch/host. The order of numbering is intentionally left to the implementer, who may base it on interface indices, port numbers, etc.. In many cases, it can be omitted, along with the at-sign (@). Current implementations, for example, use the ident field to refer to different ports on hosts participating in a LAG.

This standard is living and open to change at any time; perhaps it can be expanded if our switches change.

Network Statistics

Network statistics are collected by Management and are printed conveniently on STAT.

Visit http://stat.cosi.clarkson.edu/cacti (with default username and password for the labs) and then jump around the tree

Network Map

Cosi-network.svg