Orbit > Building Orbit > PXE

PXE on Orbit

To bootstrap the testbed nodes, in fact any server in the system, we use ​PXE to have the nodes load an image which contains a nodeagent and a ​frisbee client.

PXE requires the following support:

1. A ​DHCP server pointing to an TFTP server
2. A ​TFTP server proving the following:
   1. A network bootstrap program (NBP)
   2. A file containing instructions for the NBP
   3. A boot image
3. A PXE grid service

Ideally the later referenced "orbit-pxe-server" apt package should include dependencies to all required packages and should setup everything. For instance, it should depend on gridservices and should then automatically instantiate the PXE grid service.

DHCP Server

To support PXE we need a DHCP server. This server does not necessarily need to be instantiated at the same machine as the TFTP server, but it's configuration needs to point to it.

Installing a DHCP server on Debian is as simple as:

% apt-get install dhcp

To support PXE, we need to add to /etc/dhcpd.conf something like:

option domain-name "example.com";

default-lease-time 600;
max-lease-time 7200;

allow booting;

# The next paragraph needs to be modified to fit your case
subnet 192.168.1.0 netmask 255.255.255.0 {
  range 192.168.1.200 192.168.1.253;
  option broadcast-address 192.168.1.255;
# the gateway address which can be different
# (access to the internet for instance)
  option routers 192.168.1.1;
# indicate the dns you want to use
  option domain-name-servers 192.168.1.3;
}

group {
# The address of the TFTP server that hosts the PXE image.
 next-server 192.168.1.3;
 host tftpclient {
# tftp client hardware address
  hardware ethernet  00:10:DC:27:6C:15;

# The PXE boot loader that instructs the client to load a script corresponding to its IP address.
# The script defines the kernel and memory based image to download and boot.  Think of it as a 
# network aware grub/lilo type boot loader.
  filename "/tftpboot/pxelinux.bin";
 }
}

After you have edited the dhcpd configuration file, restart it with "/etc/init.d/dhcpd restart".

Installing a TFTP server

Debian makes that easy. We prefer atftpd since its the only tftp server that has verbose output. It also gives the ability to control the number of threads which handle transfers. With 400+ simultaneous clients, a good tftp server is required.

% apt-get install atftd

Installing NBP — pxelinux

'pxelinux.bin' is a file found in the syslinux package. ​Syslinux The nitty gritty of how it works, what to do to make a machine netboot, and all of the other specifics can be found at ​Syslinux.

Installing an PXE image

The PXE client gets the address of the TFTP server through DHCP. Taking it's IP address it attempts to download a specific file containing further instructions from the TFPT server. The name of the file is created from the PXE client's IP address. For Orbit, the instructions are identical for every node and are stored in "/tftpboot/pxelinux.cfg/default.orbit-version". This file instructs the node to fetch the image "/tftpboot/initrd-orbit-pxe-version.img" from the TFTP server and boot into it.

Both files will be installed through:

% apt-get install orbit-pxe-server

Installing the PXE Grid Service

Every service on Orbit which can be controlled by the experimenter requires a grid service component. The PXE grid service must be installed on the TFTP server.

% apt-get install gridservices
% cd /etc/gridservices/enabled
% ln -s ../available/pxe .
% /etc/init.d/gridservices restart