OpenFlow

Experimentation in ORBIT

Sandbox 9 is a great option for OpenFlow/SDN experimentation.

As shown in figure below, SB9 is built around an OpenFlow capable switch, a Pronto 3290, with 11 experimentation nodes connected to it. Seven of these nodes (node1-1 through node1-7) have ​NetFPGA 1G cards, two (node1-8 and node1-9) have ​NetFPGA 10G cards, while the other two are general purpose ORBIT nodes that could be used as controllers or monitors. The switch provides the 'DATA' backplane for SB9 with the 'eth0' interfaces on each node and the 4 data ports on each of the NetFPGAs connected to it at the specific ports shown in the figure below. The control interfaces on the nodes (i.e., 'eth1' ) are connected to an external control switch not accessible to the experimenter (primarily used for node imaging and node access from console).

Figure 1: Overview of the SB9 setup.

OpenFlow Operation Modes for P-3290

The P-3290 can operate in 2 basic modes: as a L2 switching/L3 routing element using Pica8's PicOS protocol stack, or as a multi-layer virtual switch enabled by porting open vSwitch (OVS), where the control path is open to experimentation. In addition, since the U-Boot bootloader on the switch can load custom images via NFS/TFTP, one can load any compatible switch OS solutions (e.g., Indigo), presenting a third path for OpenFlow experimentation.

A. Openflow in L2/L3 Mode

As of PicOS 2.0, the switch can mix data traffic between the OpenFlow and L2/L3 networks. They call this enhancement “crossflow” mode. Individual ports can be designated either as legacy or crossflow port, where the control on the legacy ports is handled by PicOS L2/L3 stack. The control for crossflow ports can be left to one of local handling (by PicOS stack), by flow rules entered through CLI, or by establishing path to an OpenFlow controller.

Figure 2: Hardware structures used to store flow rules in the TCAM and L2/L3 crossflow modes in P-3290

There are two further distinct modes within the crossflow mode, L2/L3 mode and TCAM mode, that use different hardware lookup structures during forwarding. If you enable L2/L3 mode, all flows will be installed in the routing table and the FDB table in the hardware. If you enable TCAM mode, all flows will be installed in the TCAM table. In each of these sub-modes, there are limitations on what flow patterns can be specified - refer to the attached configuration guide for these constraints.

Preparing the switch for OpenFlow mode includes: assigning h/w resources for OpenFlow flows, enabling OpenFlow mode and enabling specific versions (e.g., 1.1, 1.2 or 1.3), setting required ports as crossflow, and finally adding the 'match' and 'action' details for each flow. As a more standard alternative to the last step of introducing flow rules through the CLI, detailed for a remote OpenFlow controller can be configured that will then receive headers/packets that are not handled locally on the designated crossflow ports. An example of each step is shown in the following excerpt from a CLI session:

-- set tcam/routing table resources

XorPlus# set interface max-acl-rule-limit ingress 400 
[edit]
XorPlus# set interface max-route-limit 6000 
[edit]

-- (globally) enable OpenFlow, and set the working mode, i.e., either L2/L3 or TCAM modes

XorPlus# set open-flow
[edit]
XorPlus# set open-flow working-mode tcam-mode 
[edit]

-- enable specific OF version, e.g., v 1.3

XorPlus# set open-flow allowed-versions openflow-v1.3 disable false
[edit]

-- set required ports in crossflow mode, with or without local control

XorPlus# set interface gigabit-ethernet ge-1/1/1 crossflow enable true
[edit]
XorPlus# set interface gigabit-ethernet ge-1/1/2 crossflow enable true
[edit]
XorPlus# set interface gigabit-ethernet ge-1/1/2 crossflow local-control false
[edit]

-- add flow rules directly from CLI

XorPlus# set open-flow flow f1 match-field ethernet-destination-address 22:22:22:22:22:22
[edit]
XorPlus# set open-flow flow f1 match-field vlan-id 100
[edit]
XorPlus# set open-flow flow f1 match-field ip-destination-address 192.168.1.0/24
[edit]
XorPlus# set open-flow flow f1 match-field ethernet-type 2048
[edit]
XorPlus# set open-flow flow f1 action output interface ge-1/1/1

-- configure controller IP/port/transport for non-local control

XorPlus# set open-flow controller contr-serv address 10.19.0.11
[edit]
XorPlus# set open-flow controller contr-serv port 6633
[edit]
XorPlus# set open-flow controller contr-serv protocol tcp 
[edit]
XorPlus# commit

2. Openflow in OVS Mode

3. Openflow through Network Boot (of custom image)

Switching Modes using Web Service

Access to Switch CLI

Access to the switch CLI while the switch is in the L2/L3 mode or the OVS mode is enabled through ssh using the following account credentials:

ssh host/IP: sw-sb-09.orbit-lab.org/10.19.0.253
username: sb9user
passwd: sb9passwd

Note that the ssh works from within sb9 only, and would therefore require the experimenter to have reserved a SB9 time slot.

While in L2/L3 mode, the XorPlus CLI under the above account grants complete control to configure the switch and also to load/store configuration files on the root file system. However, it is requested that experimenters who want to preserve their configs to copy out their configurations files using tftp, etc., to external locations and load them when required again. Below are the relevant CLI steps with further details in the attached configuration guide. Please exercise prudence and caution when saving or moving configuration files in/out of the root file system.

-- retrieve config file from external location; scp can be used in place of tftp

XorPlus> file tftp get remote-file basic.conf local-file /pica/config/sb9user/ychen.conf <server IP>
XorPlus> configure

-- load previously stored config file 

XorPlus# load ychen.conf
-- make any number of changes and commit

...

-- save current configuration changes to specific file

XorPlus# save ychen-latest.conf
Save done.
[edit]

-- transfer out config file to safe location for later use

XorPlus# exit
XorPlus> file tftp put local-file /pica/config/sb9user/ychen-latest.conf remote-file updated.conf <server IP>

External References:

• ​OpenFlow
• ​Pronto 3290 (now simply P-3290)
• ​NetFPGA 1G and 10G cards

.