NSANet is designed to segment world-facing network services (such as servers) from network devices that non-csg people have no business being anywhere near.
Although most of these interfaces (e.g. the telnet console on switches) have passwords that isn't really fantasic security and for peace of mind it would be better that the public at large can't reach them.
A pair of bastion hosts (parakeet and firecrest) provide the only interface between the outside world and NSANet. NSANet packets move between switches on a private vlan that has no IP address and thus cannot be routed to or from. The only way to get NSANet access is via a port on a switch that has been configured to the NSANet vlan.
The bastion hosts also provide services within NSANet such as DNS (there is no DHCP), console server and UPS management.
NSANet operates on 10.1.0.0/16 and devices have IP address thus:
| CIDR Range | Class of Device |
| 10.1.1.0/8 | Bastion hosts |
| 10.1.2.0/8 | Cyclades serial servers |
| 10.1.3.0/8 | APC UPSs |
| 10.1.4.0/8 | APC MasterSwitches |
| 10.1.5.0/8 | Other |
The two bastion hosts and the cyclades are connected directly to the NSANet core switch which uplinks into the DoC (physical) network as a VLAN.
Switches and Routers smart enough to do so can save their config via TFTP to a Bastion host. Since the NSANet isn't routed this takes place over the DoC network. As TFTP has no security access must be restricted in /etc/firewall.
The tftpd is run via xinetd and roots in /home/tftp. In that directory is a writable file for each switch number (200-254). TFTP will not create files; in order to write there must be an existing file which is world writable.
A cron job /etc/cron.daily/router_config_rotate calls /home/tftp/rotate_all.sh to spin the logs. The last 14 days worth of configs are kept in subdirectories.
The spun configs are world readable, thus TFTPable and are named 123/123.conf-2003-08-28 where 123 is the switch number.
Switches should be configured to upload their configuration at 222-200 minutes past two in the morning, where 222 is the switch number. Thus 146.169.11.222 would upload at 0222.
Both parakeet and firecrest provide DNS services and their DNS tables must be kept in sync. Given that the devices on the NSANet should be resonably static an automatic sync would be more trouble then it's worth.
The DNS config lives in /etc/djbdns/root/data and the format is very simple. To add a new host add a line of the form:
=hostname.nsanet.doc.ic.ac.uk:10.1.2.3:86400
The 86400 is the DNS TTL and really doesn't matter as nothing will be caching the record anyway. After editing the data file one must cd /etc/djbdns/root ; make ; svc -h /services/djbdns. And remember to make the same change on both bastions.
The bastions will only serve DNS requests to the NSANet, but will resolve anything (not just nsanet.doc.ic.ac.uk names). On the bastions it's important to give the nameservers by their NSANet address in resolv.conf.
Consoles are either accessed over serial ports physically on one of the bastions (e.g. a Cyclades Z-series card) or via a Cyclades standonly serial server. A number of programs are involved in handling serial data. For example, in the serial server case the flow of data looks like:
To connect to a console simply run console lark where lark is the name of the console you want. If you don't have access rights it will quit.
You start off without control. If you try to type anything you will be warned that you do not have control. The escape sequence is ^E and the commands are:
| Key following | Action | Description |
| ? | Print help | |
| t | Take control | Fails if another user has control |
| d | Drop control | |
| s | Snatch control | Always succeeds. "Force attach" in conserver speak |
| f | Flush buffer | Delete the outgoing buffer that conserv is holding for you. Doesn't clear the kernel socket buffer. |
| S | Show stats | Gives uptime and traffic counts |
| h | History | Dump the last 2k of data |
| w | Welcome | Show the welcome data again. Includes user list. |
To send a break type ^] b b in quick succession.
The following lines should be in the GRUB config (/boot/grub/menu.lst):
serial --unit=1 --speed=38400 terminal --timeout=10 serial console
The unit=1 option selects ttyS1 and you may have the change this for a given server. This is correct for the Intel RJ45-serial servers at least.
The options console=tty0 console=ttyS0,38400 should be passed to the kernel also.
/etc/inittab should have a line like 7:12345:respawn:/sbin/agetty -n -L -l /bin/serialbash 38400 ttyS1. The file /bin/serialbash should be as follows:
#!/bin/sh exec /bin/bash --rcfile /etc/bashrc-serial
And /etc/bashrc-serial is:
sleep 1 /bin/stty sane /usr/bin/reset tset vt100 HOSTNAME=`/bin/hostname` TTY=`/usr/bin/tty` echo This is $HOSTNAME export PS1='\u@\h # '
Deveation from these exact instructions will condem you to subtle-terminal-bug-hell.
New Intel servers have an RJ45 port on the back that is labeled as a serial port. It is wired as a Sun Netra serial port and you must connect them with the correct cable. These cables are purple with a blue and an orange gland. Plug the blue end into the Cyclade and the orange end into the server. The wiring for these cables can be found in a Cyclades manual.
For devices with a real DB9 serial connector you will require a converter.
Null-Modem wireout (for Intel servers):
| D-Type | Wire Colour | RJ45 |
| 8 | Brown | 1 |
| 1 | White | 2 |
| 2 | Black | 3 |
| 5 | Red | 4 |
| 7 | Green | 5 |
| 3 | Yellow | 6 |
| 4 | Blue | 7 |
Pass-Thru wireout (for Chaparrel RAID controllers)
| D-Type | Wire Colour | RJ45 |
| 7 | Brown | 1 |
| 1 | White | 2 |
| 3 | Black | 3 |
| 5 | Red | 4 |
| 8 | Green | 5 |
| 2 | Yellow | 6 |
| 6 | Blue | 7 |
APC UPSes have their own pinout. Get a CPC part number CN04265 (black DB9->RJ45, male) and connect the wires thus:
CPC CN04265's are coloured like this:
| Pin Number | Colour |
| 1 | Blue |
| 2 | Orange |
| 3 | Black |
| 4 | Red |
| 5 | Green |
| 6 | Yel |
| 7 | Brown |
| 8 | White |
The Cyclades run Linux and see their serial ports as normal ttySx devices. A process called portslave handles the exposure of the serial ports to the outside world. In NSANet configuration they are mapped to TCP ports 7001-7032. These ports are firewalled at the Cyclades INPUT interface against TCP SYN packets. Thus the only way to get at these ports is via an SSH tunnel.
For logging reasons the connections to the Cyclades are always open and the Cyclades only allow a single connection to a given serial port. The sshd on the Cyclades are configured for both TCP keepalive and SSH heartbeat so a failed bastion server shouldn't leave ssh processes locking the Cyclades up for too long.
Put the Cyclade on a serial port and bring up minicom. They run 9600 8N1. Login as root (hopefully no password) and bring up eth0 with ifconfig. Set the root password.
Get zImage_ts_137.bin and boot_alt_ts_201.bin from /vol/source/cyclades-firmware/v1.3.7. SCP the two files to root@cyclade:/proc/flash/xyz where xyz is either zImage or boot_alt. This will take a while as the poor Cyclade is pretty slow.
Over serial type reboot and hope. You should see a message saying Restarting syst and the hardware startup will trigger. You can hit esc to abort the memory check. Always enable the watchdog and boot from firmware if it asks. The root password is now tslinux.
Over serial, bring up eth0 again and mkdir /root/.ssh. SCP cyc_authorized_keys_file in root@parakeet:/root/.ssh to root@cyclade:/root/.ssh/authorized_keys on the cyclade.
Add it to the NSANet DNS (see above).
On a bastion host edit serial_devices to add a device on the new Cyclade (make one up if need be) and reconfigure. This will setup the rest of the Cyclade and save config to flash.
The config.py script, in /etc/conserv reads in four config files, whose filenames are given on the command line. If you just run python config.py (i.e. without arguments) it will give a usage telling you the config files that it expects.
In the same directory there is a Makefile which will do everything you want if you just type make.
The config files are serial_devices, serial_types, access and consoles. Every serial device should be configured in serial_devices and the type column of which must match up against a row in serial_types. Those devices which are exposed via conguardian (UPSs and consoles) must be listed in consoles and the access_tag column of which references a line in access.
The lines in access consist of a tag name, followed by the members of that tag. Members can either be system usernames or previously defined tags.
The config.py script keeps cyclade configs in /var/cyc/config and diffs the generated config with the cached one. If there is a difference the new config is scp'ed to the Cyclade and saved to flash. The files /var/consoles/start.sh and /var/consoles/auth.sh are also written. The first starts all the consoles up and the second sets the access controls. The second can be run whenever the access controls are changed, and the first can be run to start any consoles that have died.