SB9

SB9 is dedicated to wired networking and cloud computing experimentation. Nodes in sandbox 9 don't have wireless interfaces.

Sandbox 9 consists of 8 servers, each with the following specifications:

• Server model: Dell R740XD
• CPU: 2x Intel Xeon Gold 6126
• Memory: 192GB
• Disk: 256gb ssd
• Network: 2x 25gbe mellanox connect-x4 lx
• Network: 2x 100gbe mellanox connect-x5 lx (pending, shares 1x pcie3.0x16 slot)
• Network: 1x 10gbe (pending)
• Accelerator1: Nvidia V100 16gb
• Accelerator2: Xilinx Alveo U200 FPGA (2x 100gbe ethernet onboard)

They are connected to the following P4 Capable Switch: Edge Core Wedge100BF-32x

• Ports: 32x 100g ports, each can break out to 2x 50, 2x 40, 4x 25, or 4x 10g ports.
• OS: Azure SONIC operating system, or OpenNetworkLinux
• Management: In band, out of band, or via onboard BMC.

Procedures

Manual install of new switch os: Currently this must be done by a user with admin access. A service will be provided to automate this operation.

1. download image and change symlink on repo1 at path /tftpboot/os/switch/100bf-32x/current-os
2. Connect to BMC, and run ONIE following steps at ​https://github.com/opencomputeproject/OpenNetworkLinux/blob/master/docs/GettingStartedWedge.md
3. For SONIC, follow ​https://github.com/Azure/SONiC/wiki/Quick-Start
4. Set up mapping of serial lanes to ports

Old

All hardware below this line has been retired. The information is preserved for reference.

SB9 is dedicated to ​OpenFlow experimentation (NOTE: Nodes in SB9 don't have wireless interfaces!). As show in Figure 1, Sanbox 9 is built around an OpenFlow capable switch with 12 nodes, 7 of which are equipped with ​NetFPGA cards, 2 nodes with ​NetFPGA-10G card and 2 general purpose ORBIT nodes and a sandbox console. In addition, there is a fast cache machine, with a 10g connection and a ssd storage array fast enough to saturate that link.

Figure 1: SB9

The switch labeled 'sw-sb-09', a ​Pronto 3290, provides the central connectivity backplane for 'DATA' interface of all hosts/NetFPGAs in the sandbox. Each host (node1-1..node1-12) is connected to the sw-sb-09 switch through one 1GbE data connection on the interface 'eth0' or 'exp0'. As is the case with the rest of ORBIT nodes, a second GbE interface ('eth1' or 'control') of each node, that is used exclusively for experiment control (incl. ssh/telnet sessions), is connected to an external control switch outside of sandbox.

The first 7 hosts (node1-1..node1-7) each contain a 4x1GbE NetFPGA installed on a PCI slot. Each NetFPGA has 4 connections to the top switch, corresponding to its 4-GbE ports nf2c0-nf2c3.

Nodes node1-8 and node1-9 each contain a 4x10GbE NetFPGA installed on a PCI Express slot. top switch while the remaining two are directly connecting the two nodes.

Node1-12 has eth4 connected to the pronto switch at 10g, with eth5 currently unconnected. It also has 2 unconnected 10gbase-T ports, enumerated as eth2 and eth3.

The Pronto switch is a OpenFlow enabled switch and can be run in native or OpenFlow mode by controlling its boot configuration. When in native mode, it runs the Pica8 Xorplus switch software while when it is in OpenFlow mode it can run either stock Indigo firmware or experimenter provided OpenFlow image for Pronto switches. The mode of operation is controlled by the network aggregate manager service.

Switchport #	Node	Device	Interface #
1
2	1-1	NetFPGA	Port4
3	1-1	NetFPGA	Port3
4	1-1	NetFPGA	Port2
5	1-1	Card	Data
6
7	1-2	NetFPGA	Port4
8	1-2	NetFPGA	Port3
9
10	1-2	Card	Data
11
12	1-3	NetFPGA	Port4
13	1-3	NetFPGA	Port3
14	1-3	NetFPGA	Port2
15	1-3	Card	Data
16
17	1-4	NetFPGA	Port4
18	1-4	NetFPGA	Port3
19	1-4	NetFPGA	Port2
20	1-4	Card	Data
21
22	1-5	NetFPGA	Port4
23
24	1-5	NetFPGA	Port2
25	1-5	Card	Data
26
27	1-6	NetFPGA	Port3
28	1-6	NetFPGA	Port1
29	1-6	NetFPGA	Port2
30	1-6	Card	Data
31
32	1-7	NetFPGA	Port4
33	1-7	NetFPGA	Port3
34	1-7	NetFPGA	Port1
35	1-7	Card	Data
36	1-11	Onboard	Data
37
38
39	1-10	Onboard	Data
40	1-8	Card	Data
41
42
43
44
45	1-9	Card	Data
46
47
48
te-1/1/49	1-12	Card	Data
te-1/1/50	?	NetFPGA	?
te-1/1/51	?	NetFPGA	?
te-1/1/52	?	NetFPGA	?
Eth1	Management	SW-SB-02	44
Eth2	Management
Aux
Con	Management	S-ORBW	Serial1