PXE Aggregate Manager

PXE Aggregate Manager configures tftp service to enable network loading of the ram based image that is used for imaging (boath loading and saving of the disk images) on the node. Once ram based image (orbit-pxe) is loaded it runs Resource Controller (RC) that will wait for the appropriate commands from the Experiment Controller (EC). The PXE AM will, on demand from EC (or any other AM) make a symbolic link of the node's IP address in hex pointing to the appropriate (inventory based) configuration file on the TFTP server. By default PXE configuration is to have the node boot off its local hard disk as specified in /tftpboot/pxelinux.cfg/default :

default harddisk

label harddisk
  localboot 0

Usage Example

Experiment controller (EC) issues:

http://pxe:5054/pxe/setBootImage?node=node1-2.sb8.orbit-lab.org

The PXE AM in turn, queries the inventory AM to find out what is the approrpiate boot configuration for node1-2, and creates a symbolic link for the respective IP address in hex pointing it to a returned configuration file. For example, if the default boot configuration is obrit-2.0.0 and the node has an IP address of 10.18.1.2 (the case of sandbox8, node1-2) the respective IP address in hex is 0x0A.12.01.12 and as such its symbolic links' filename will be 0A120112.

/tftpboot/pxelinux.cfg/0A120112 --> orbit-2.0.0

The configuration file specifies kernel and disk layout…

DEFAULT boel

LABEL boel
KERNEL linux-orbit-pxe-2.6.20.4
APPEND rw init=/sbin/init initrd=initramfs-orbit-pxe-2.0.0.gz console=tty0 console=ttyS0,9600
DISPLAY message.txt
PROMPT 1
TIMEOUT 1

After the symbolic link request was created, the EC will the reboot the node and, based on this example, the node will load linux-orbit-pxe-2.6.20.4 with initramfs-orbit-pxe-2.0.0.gz as its initramfs cpio archive. The kernel then boots and finally calls /etc/init.d/rcS:

#!/bin/sh
NETDEV_0="eth0"
NETDEV_DHCP="eth1"
UDHCPC_SCRIPT="/etc/udhcp.script"
UDHCPC="/sbin/udhcpc"
IP_CONFIG="/bin/ip"

mount -t proc proc /proc
# pts requires a directory for multiple pts's:
mkdir /dev/pts
mount -t devpts devpts /dev/pts

ifconfig lo 127.0.0.1

# Get network up
$IP_CONFIG link set $NETDEV_0 up
$IP_CONFIG link set $NETDEV_DHCP up
$UDHCPC -i $NETDEV_DHCP -s $UDHCPC_SCRIPT
# No need to request an IP via DHCP on NETDEV_0
# If you do add it make sure you'll get an answer or
# be sure to background the job
# $UDHCPC -i $NETDEV_0 -s $UDHCPC_SCRIPT &

# enable logging (if the feature is built into busybox):
# Note that nodeagent4 doesn't use syslog anyway so you don't
# really need it unless you want to monitor some other daemons
if [ -f /sbin/syslogd ]; then
        /sbin/syslogd
else
        echo "Syslog daemon not found"
fi

# You don't need this if you have serial access ;)
if [ -f /usr/sbin/telnetd ]; then
        /usr/sbin/telnetd
else
        echo "telnet daemon not found"
fi
cat /.orbit_image
# Start omf-resctl-5.4
omf-resctl-5.4&

After this RC will just wait until EC issues it further commands.

Details on PXE can be located ​here.

---

Notes

PXE service is heavily dependent on a properly configured DHCP and tftpboot service. We use the following versions:

ii  tftpd-hpa                            5.2-1ubuntu1                      HPA's tftp server
ii  isc-dhcp-server                  4.1.ESV-R4-0ubuntu5.9             ISC DHCP server for automatic IP address assignment

---

Installation and Configuration

The PXE service is packaged in the omf-aggmgr-pxe-5.4 package. It can be installed form the package repository with

apt-get install omf-aggmgr-pxe-5.4

Instructions on how to add the Orbit software repository to your list of apt-sources can be found ​here.

Once installed the service is configured by TODO.