The IP8800 Packet Switch
The IP8800/S3640 is a smart switch built by NEC capable of running firmware compliant with OpenFlow networking. WINLAB utilizes IP8800 switches in its experimental network infrastructure. This is a quick-start guide (hopefully) suitable for anyone trying to learn how to deploy the NEC IP8800 on the network as an OpenFlow device.
Index
Before we begin...
I Introduction
1.1 Connection via serial
1.2 User modes
1.3 Finding Information
1.4 device information lookup
1.5 UNIX commands
1.6 Powering off
2.1 Contexts
2.2 VLANs and VLAN Interfaces
2.3 Ports
2.4 Telnet
2.5 SSH
III Miscellaneous Configuration Tasks
3.1 ACLs and SNMP
3.2 DHCP Relay Agent
3.3 DHCP Server
3.4 Port Mirroring
3.5 Users and Passwords
3.6 DNS configuration
4.1 What is OpenFlow?
4.2 Outline
4.3 Configuring the switch
4.4 Preparing the SD card
4.5 Flashing the switch
4.6 OpenFlow Commands
4.7 Relevant to ORBIT
Before we begin…
A few things before moving onto business. Firstly, Here are the assumptions that are made in this handbook:
- You are using the NEC IP8800/S3640-48T2XW switch. Commands vary even amongst similar lines of devices.
- You are familiar with, and running some form of *nix on your PC (We are, for the most part, using Ubuntu unless specified)
- You have a serial port on your machine, and have either minicom or kermit installed on your machine.
References
The full IP8800 operations manual can be found online. Then why am I taking my time to make this handbook? For one, the online manual is in Japanese. Even if you are literate in Japanese, the manual is quite extensive, and doesn't cover the OpenFlow side of things. So, to sum things up — the purpose of this guide is to provide quick reference to the switches' basic functionalities so that no-body has to dig through the handbook to find out, say, how to safely shut the switch down. But, for those of you who want to check out the whole shebang, here are the links:
- Vol. 1: http://www.nec.co.jp/ip88n/s36_sw/html/cfguide/index.html
- Vol. 2: http://www.nec.co.jp/ip88n/s36_sw/html/cfguide2/index.html
- Vol. 3: http://www.nec.co.jp/ip88n/s36_sw/html/cfguide3/index.html
The switches may also come with 2 CDs containing the English translated version of the manual. The OpenFlow manual comes with the firmware.
Conventions
Text in console font and in quotations are commands that can be entered at the command line. When in quotations, the quotes should be omitted. Words surrounded by carets are various options for a command. If delimited by pipes (|), one out of the several commands separated by pipes can be entered in that spot. For example:
interface gigabitethernet <src port> <tx|rx|both>
Here, <src port> is a value between 0/1 and 0/48, and can be followed by keyword tx, rx, or both.
Each section outlined in the index starts with a short overview, followed by step-by-step instructions (if applicable), and ends with troubleshooting error messages that are associated with each.
I Introduction
Many "Smart" switches, including the IP8800 series switches, must be configured using the Command-line-interface(CLI). This usually involves a serial, telnet, or ssh session through a terminal. This chapter explains how to connect to the switch via serial, and covers some basic commands and operational tasks to get started with these switches.
1.1 Connecting via serial
Telnet is not enabled by default on the switches. Therefore, initial configurations must be done through a serial connection. The steps below describe how to establish a serial connection with the switch using kermit, as its default settings allow it to connect without any modifications. Few laptops have serial ports these days, so have a desktop or a USB-to-serial converter handy.
If not using kermit, the following settings are required if settings must be modified:
* speed: 9600 * data bits: 8 * stop bits: 1 * Flow control: none * Parity: none
Steps.
- Name your switch. People names are easier to remember than IP addresses or machine-like names i.e. as01-hh-alex, and are easier to use in discussion if everyone agrees on names. Note, the names Bob, Nancy, and Andy are already taken.
- Establish a connection. Connect your PC serial port to the port labeled "console" on the switch using a RS-232 cable, and issue the following commands as root from a terminal:
# kermit C-Kermit>set line /dev/ttyS0 C-Kermit>set carrier-watch off C-Kermit>connect
The following is also valid:
kermit -l /dev/ttyS0 set carrier-watch off connect
If all goes well, you should then see something like below. If the login prompt doesn't come up, hitting the enter key should bring it up.
Connecting to /dev/ttyS0, speed 0 Escape character: Ctrl-\ (ASCII 28, FS): enabled Type the escape character followed by C to get back, or followed by ? to see other options. ---------------------------------------------------- login:
- Login. The default username is "operator", and the password is blank. Once logged in, you will be brought to the CLI. Commands are entered at the caret.
login: operator Copyright (c) 2005-2008 ALAXALA Networks Corporation. All rights reserved. >
- exiting.
Ctrl-\ qends the kermit session. The commandexitorlogoutwill just log you out of the switch, but will not end the session.
Troubleshooting serial connections
- If using kermit, make sure you are root when you start up your connection. minicom does not require you to be root.
- Kermit will balk if you try to log in while there is a session going on. Make sure that no one else is logged in via serial, and that you haven't forgotten to log out during a previous session.
1.2 User modes
Enabled Mode
You are in initially in user mode when you log into the switch. Only a limited set of commands for status lookup are accessible from user mode. In order to gain full access to device (e.g. for configuration) you must be in enabled mode. For all intensive purposes, this is equivalent to superuser mode in *nix. The command enable allows a user to enter enabled mode. The whole command need not be entered at the prompt, as the switch is capable of command interpretation.
> ena <<---whole command is 'enable' -- the switch assumes and autocompletes commands. #
Note that once in enabled mode, the caret becomes a pound. Command "exit" exits you from enabled mode (and any other modes or contexts, such as configuration mode explained next).
Configuration Mode
Once in enabled mode, switches can be configured by entering configuration mode. All configurations, from ACLs to port speed, are done from this mode, or, as we may interchangeably call, context. Configuration mode may be entered via the command configure.
# configure (config)#
We will be delving further into the details of configuration starting with chapter II of the guide.
1.3 Finding Information
Typing ? at the prompt will display all of the basic commands that are available. Typing ? after a articular command will display the available flags.
sw-sb09> reload ? -f Restart without confirmation dump-image Memory dump is extracted and preserved no-dump-image Memory dump is not extracted and preserved stop Stop, but do not restart <cr>
Here, <cr> means hit return.
An alphabetized list of commands and their usage can be found here, although this is also in Japanese.
The switch also recognizes some UNIX commands (see the UNIX commands section for details). For such commands (and OpenFlow commands such as showswitch), "command --help" or the flag -h may also be used to get information about a certain command.
1.4 Device information lookup
This is primarily done through the show command. Some useful lines:
show configuration- all current configurations, e.g. hostname, VLANs, ACLs, portsshow port statistics- port status (interface name, up, down, ect.)show port vlan- list all port-to-VLAN associationsshow version- System information e.g. serial number, OS version
For a complete list, do "show ?".
In configuration mode, the show command will display the configurations for the current device element you are configuring:
(config)# interface gigabitethernet 0/27 <<---entering port 0/27 config prompt (config-if)# show interface gigabitethernet 0/27 switchport mode access switchport access vlan 4 !
Again, more details about the hows of configuration can be found in chapter II of the guide.
1.5 UNIX commands
The core OS of the switch is NetBSD 2.0. Therefore, some *nix commands can be used at the CLI. See /usr/local/bin, /usr/local/sbin, /usr/bin and /usr/sbin for a list of available commands. For example, the up arrow key will let you access previously issued commands, as will !.
some commands
Note, enabled mode is not true superuser (user "admin", not "root"), so your actions are quite restricted. However, some useful commands are:
- find <rootdir> -/name <targetname> : find a *nix command
- /usr/bin/clear : clear the terminal screen
- /usr/bin/who : list users logged in
- /sbin/sysctl : system specs/status
- /sbin/dmesg : display system message buffer
- /sbin/mount : show mounted devices (good for troubleshooting SD card issues)
- scp : better (more intuitive, I think) than ftp
If you feel compelled to read them, try the Online NetBSD man pages. The built-in man pages are not very functional.
text editors
If you desperately need to, and can't do it any way else, things (e.g. your OpenFlow config file, discussed in Chapter IV) can be edited using ed. Be forewarned, this is not for the faint-hearted.
1.6 Powering off
Yanking the power on a switch suddenly can cause file corruption, so we want to properly stop all processes before cutting the power. To shut down a switch, issue the command reload stop. This halts the system (stops all processes), but does not power the hardware off. After the system halts, the rom prompt will appear. At this point, you'd have to manually turn the switch off.
sw-sb09-da# reload stop Dump information extracted? (y/n): y old dump file(rmdump 07/21 21:16) delete OK? (y/n): y sw-sb09-da# ROM 00.03.52 Rom:
II Basic Configuration Tasks
This chapter covers the basics of switch configuration. We will first take a look at some of the more important contexts available from configuration mode. The overview will be followed by a look at lower layer constructs (ports, VLANs, IP addresses) with respect to the switch, and how to configure these elements. This chapter will end with the configuration of SSH access so that we can access the switch over the network securely.
2.1 Contexts
Entering Contexts
All configurations are done by entering various contexts from the (config)# prompt. Some contexts we want to focus on are:
vlan n- create/configure standard VLANs, n being the VLAN ID (some number)interface vlan n- create/ configure a virtual interface with both Layer 2 and 3 characteristicsinterface gigabitethernet 0/n- configure specific port, n being number between 1 and 48line <vty|console> vtyrange<0-b>- configure telnet or console parameters, respectively
The details of each context will be introduced in the following sections.
when you enter a context, the prompt becomes "(config-if)#." Note, however, this prompt doesn't indicate what context you are using (i.e. which port you are setting up, what VLAN, ect); you just have to keep track of what you are doing.
(config)# interface vlan 8 <<---start by entering the 'interface vlan' context !(config-if)# !(config-if)# interface gi 0/3 <<---switch to 'interface gigabitethernet' context !(config-if)# <<---note how prompt still shows up as "config-if"
If you forget where you were in a context (i.e. which vlan you were configuring, ect.), typing "show" will display all of the configurations saved for that specific context (hopefully giving you enough information to let you recall what you were trying to configure).
As you can see above, when there are unsaved changes in the configurations (e.g. a new vlan interface created, as in the case above), you will see a "!" before the prompt. This is there to remind you that you've made changes that must be saved if you want them to survive a reboot. The command "save" intuitively saves the settings.
undoing configurations.
Re-running a command with "no" prepended to it will usually undo the configuration. Exceptions to this rule will be mentioned in the appropriate sections.
exiting/ logging off.
The command exit lets you exit from contexts and enabled mode, or lets you log out of the switch if you were just in user mode.
!(config-if)# save (config-if)# exit (config)# exit #
2.2 VLANs and VLAN Interfaces
Index
This section is divided into the following sub-sections.
2.2.1. Background
2.2.2. VLAN creation
2.2.3. Naming VLANs
2.2.4. Associating VLANs to ports
2.2.5. IP address assignment
2.2.6. VLAN deletion
2.2.1. Background
VLANs are a technology defined by IEEE 802.1. In terms of the switch, VLANs allow a switch to group the hosts it sees on its port based on their L2 addresses. The L2 addresses are associated with the port that it is seen on; therefore, the port that the host is attached to determines which group, or VLAN, the host belongs in.
The "vlan" context allows you to create and configure dot1Q VLANs that can be associated with ports. The "interface vlan" context allows you to configure network layer features of a VLAN by tying it to a virtual interface on layer 3. The latter context is typically used to configure IP addresses associated with the switch.
2.2.2. VLAN creation
The switch comes with VLAN 1 by default, with all ports associated with it. New VLANs can be created by entering the vlan context for a non-existent VLAN ID from the config prompt. Entering the interface vlan context for a nonexistent VLAN will also do the same thing.
creating multiple vlans. Multiple VLANs can be created at once with the command vlan <vlan range> <vlan IDs>. The interface range vlan context, however, cannot be used in this manner; The range version of the interface vlan context can only be used to configure groups of VLANs that already exist. The following error will be thrown if you try to apply the context to a non-existent VLAN, in this case, VLAN 7:
(config)# interface range vlan 7-38 interface : Not found VLAN-ID <7>.
more information on the interface vlan context can be found in section 2.2.5.
Example. The following illustrates typical steps for creating VLANs. We first generate VLANs 1-8, then 11-48, and finally, 50:
(config)# vlan 1-8 !(config-vlan)# vlan 11-48 !(config-vlan)# vlan 50 !(config-vlan)# save (config-vlan)#
return to section index
return to main index
2.2.3. Naming VLANs
Names are a good way to keep track of the various VLANs you create. VLANs are named through the vlan context with the name command.
(config-vlan)# vlan 2 (config-vlan)# name "Internal"
2.2.4. Associating VLANs to ports
Port VLAN membership is configured through the interface gigabitethernet <port> context using the switchport access vlan <VLAN ID> command.
(config)# interface gigabitethernet 0/25 (config-if)# switchport mode access (config-if)# swi acc vlan 3 <<-- full command is "switchport access vlan 3"
Here, "switchport mode access" specifies that the traffic to and from this port will not be tagged. More on port configuration is discussed in section 2.3.
return to section index
return to main index
2.2.5. IP address assignment
VLAN interfaces are abstractions that allow you to configure higher layer services and protocols that act upon ports based on their VLAN membership. One way you'd find yourself using them is to assign a management IP address to your switch.
sw-sb09-da(config)# interface vlan 1 sw-sb09-da(config-if)# ip address 172.19.10.100 255.255.0.0 !sw-sb09-da(config-if)#
The commands above assign the switch an IP address of 172.19.10.100 with subnet mask 255.255.0.0. As this is assigned to VLAN 1's VLAN interface, this address is only visible to devices on VLAN 1.
undoing static IP assignments to virtual interfaces
When you have IP-helpers configured, you want to type "no ip helper-address" at the CLI before trying to remove the IP address or else you get this warning:
interface vlan 3 ip address 192.168.5.5 255.255.255.0 ip helper-address 192.168.5.1 ! (config-if)# no ip address 192.168.5.5 255.255.255.0 interface : IP interface is not defined.
2.2.6. VLAN deletion
Deleting a VLAN
Any configurations involving a VLAN you want to remove has to be undone before deleting the VLAN. not doing so will result in the following error when you issue the no vlan <n> context:
(config)# no vlan 4 vlan : Can't delete this configuration referred by other configuration.
In this case, VLAN 4 has port 0/27 associated with it. Undoing the association allows you to delete the VLAN without issues.
(config)# int gi 0/27 (config-if)# no sw acc vlan 4 !(config-if)# exit !(config)# no vlan 4 <<-----second attempt !(config)#
return to section index
return to main index
2.3 Ports
Index
The following subsections are found here:
2.3.1. Background
2.3.2. port modes
2.3.3. VLAN association
2.3.4. Configuring trunk ports
2.3.5. port speed
2.3.1. Background
There are 48 standard RJ-45 ports and 2 fiber ports on the switch recognized as 'geth' (1Gbs) and 'tengeth' (10Gbps) by the device, respectively. We focus only on the 'geth' interfaces. The context to configure these ports is interface gigabitethernet <port>, where <port> is a value between 0/1 and 0/48. interface range gigabitethernet <port> allows you to configure multiple ports at once.
2.3.2. port modes
The switchports are access ports by default. Access ports restrict L2 broadcasts by limiting the ports that get flooded to those in the same VLAN affiliation as the port from which the frames originated. The switchport mode <port type> context is used to change the manner in which traffic to/from certain ports are handled. Off the bat, 2 are important:
switchport mode dot1q-tunnel- frames from the port will be tagged with a VLAN ID when it hits the trunk. If one port is set to this mode, it is assumed that all access ports are of this mode, regardless of whether you've touched them or not.switchport mode trunk- Sets the port as a trunk port. Frames hitting the port will be tagged with a 2-byte VLAN ID unless a native VLAN.
See section 2.3.3 for info on dot1q ports and trunk configuration.
2.3.3. VLAN association
Traffic from a port associated to a certain VLAN is only visible to other ports associated with the same VLAN (unless we're talking about trunk ports or using routers to bridge VLANs). The context switchport access vlan <n> is used to associate the port to a certain VLAN of ID <n>; See section 2.2.4 for an example.
return to section index
return to main index
2.3.4. Configuring trunk ports
With "access mode" ports, VLANs with the same VLAN ID that are on different but connected switches would be recognized as different VLANs despite having the same VLAN ID. In order to allow the two same-ID VLANs to be treated as one, we need 1) ports set to dot1q-tunneling mode, and 2) trunk ports connecting the two switches.
- Configure the trunk port. If ports 0/45-48 are to be trunk ports allowing the assage of frames from VLANs 3,27,and 28 (Remember, if you are just configuring one port, you can always use the singular "interface gigabitethernet" context.):
!(config-if)# interface range gi 0/45-48 !(config-if-range)# switchport mode trunk !(config-if-range)# switchport trunk allowed vlan 3,27,28
- "interface range gi 0/45-48" - configure all four ports at once with "interface range" context
- "switchport mode trunk" - make the ports trunk ports
- "switchport trunk allowed vlan 3,27,28" - allow the trunk to only carry frames from VLANs 3,27,or 28 (frames from these VLANs will get tagged)
Some extensions of the "switchport trunk allow" context."
As per the switch's help output:
sw-sb09(config-if-range)# switchport trunk allowed vlan ? <vlan id list> <List style>. 1 to 4094. ex. "10-20", "30,40" add add the VLANs to use this port remove remove the VLANs to use these port
Summarized:
- "switchport trunk…vlan add <VLAN IDs> - add VLANs to your current "allowed VLANs" list.
- "switchport trunk…vlan remove <VLAN IDs> - remove VLANs from your current "allowed VLANs" list
- Associate the trunk with a Native VLAN. Native VLANs allows hosts that do not understand VLAN tagging to function on a network implementing trunking (IEEE 802.1q). In context of the switch, the native VLAN is much like a trunk port version of an interface VLAN; It essentially assigns an IP address and VLAN to the switch's trunk port(s), and provides a means to transport non-tagged frames on the trunk.
!(config-if-range)# switchport trunk native vlan 1
- "switchport trunk native vlan 1" - make VLAN 1 the native VLAN for trunk port 0/48 (any frames from VLAN 1 will pass through the trunk untagged)
By the way, this is the configuration for interface VLAN 1, which should be done ahead of time:
(config-vlan)# interface vlan 1 (config-if)# ip address 172.16.19.253 255.255.0.0
If you now go into a context for one of your trunk ports and do a "show", your configurations should look like this:
!(config-if-range)# interface gi 0/48 !(config-if)# show interface gigabitethernet 0/48 switchport mode trunk switchport trunk allowed vlan 1,3,27-28 switchport trunk native vlan 1
But finally…
- Configure the ports associated with the trunked VLANs. The steps are same as configuring the ports for regular VLANs, except instead of "switchport mode access" we specify "switchport mode dot1q-tunnel" in this case:
(config)# int ran gi 0/13-22 (config-if-range)# sw mo dot1q-tunnel !(config-if-range)# sw acc vlan 3 !(config-if-range)# save <<<-----VERY important if you want the configurations saved!
return to section index
return to main index
2.3.5 port speed
By default, the switch autonegociates. You can change this by going into configuration mode for each port, and changing settings.
- Shut port(s) to configure down. Shutting the ports down before configuring them allows the changes to take immediate effect upon bringing them back up.
sw-sb09(config)# interface gi 0/13 sw-sb09(config-if)# shutdown !sw-sb09(config-if)# 02/06 03:35:48 E3 PORT GigabitEthernet0/13 25011100 1350:000000000000 Port disabled administratively.
It is normal to see an output message when you take down ports.
- Set speed. The sub-context "speed" allows you to configure the port speed. The syntax is the following:
speed auto [10|100|1000]- autonegociate. If a speed is specified, the port will be shut down by the switch if the host refuses to negotiate to the set speed.speed [10|100|1000]- force port to run at a specific speed, no autonegociation
Where the numbers denote Mbps. Here, we configure the port to go at 1000Mbps (1Gbps), and bring it back up:
!sw-sb09(config-if)# speed auto 1000 !sw-sb09(config-if)# no shutdown
return to section index
return to main index
2.4 Telnet
This section covers the steps needed to set up telnet access to your switch. Replace the VLAN ID's, IP addresses, and other parameters to fit your deployment. Telnet is not very secure, as it sends everything using plain-text. For a more secure connection method, take a look at SSH.
- Create a VLAN for telnet via the "vlan" context. For clarity (and good practice), we name our VLAN "telnet" so we know what it's used for.
(config)# vlan 4 !(config-vlan)# name "telnet"
- Configure IP for the telnet VLAN's VLAN interface.
!(config-vlan)# interface vlan 4 !(config-if)# ip address 192.168.10.1 255.255.255.0
- Enable telnet logins to the switch using the "line vty" context. The numbers following the context denote the range of virtual terminals that the users have access to for telnet. Here we set the values to "0 2", meaning that 3 users can simultaneously telnet to the switch.
!(config-if)# line vty 0 2 !(config-line)# exit !(config)#
Using Telnet
Once you've set up telnet on your switch, you can access the switch with the IP address you've assigned it during step 2 above. So for this config you'd issue the following command from the terminal:
$ telnet 192.168.10.1
The username and password are the same for this prompt once you connect.
2.5 SSH
Unlike Telnet, SSH encrypts messages to discourage eavesdropping. You must use a password protected local user account in order to access a switch via SSH. Here we cover the configuration of password-based SSH access. Other modes of authentication (e.g. public key) are also available but won't be covered here. A detailed SSH configuration guide may be found at the NEC website.
- Create a user account with
adduser, as per this section. - From
configuremode, enable SSHv2:ip ssh ip ssh version 2 ip ssh authentication password
The first line enables SSH access to the switch by any local user with a password. The second line specifies the useage of SSH version 2, and the last, to use password-based authentication.
III Miscellaneous Configuration Tasks
This section covers some odds-and-ends that you may or may not need in terms of basic switch deployment, and that didn't really fit nicely with the past two sections. These bits tend to be a bit 'gorier' in terms of switch administration, and may require stuff covered in the past sections.
3.1 ACLs and SNMP
Index
The following subsections are found here:
3.1.1. Background
3.1.2. Configuring ACLs
3.1.3. Configuring SNMP
3.1.4. Accessing SNMP data
3.1.1. Background
Access Control Lists (ACLs) dictate who and what can access various services provided by the device. One such service is Simple Network Management Protocol (SNMP), which allows you to look up and modify device information across the network. So - here we cut to the chase and cover both at the same time. Firstly, some references for SNMP:
- SNMP, courtesy of Wikipedia: http://en.wikipedia.org/wiki/Simple_Network_Management_Protocol
- MIB references: http://www.alvestrand.no/objectid/
3.1.2. Configuring the ACL
Use of SNMP (Simple Network Management Protocol) is not set by default; you must create access lists with IP addresses allowed to request information as a manager device. ACls are configured through the access-list context. The (basic) syntax is the following:
access-list <lid> <seq> <permit|deny> <inet|any> <netmask>
where
- lid - list ID, a value between 1 - 2699
- seq - sequence number, a value between 1 - 4294967294 (can be omitted)
- inet - IP address of host or network address
- netmask - subnet mask, 0.0.0.0 for hosts
no access-list <lid> deletes all entries with a particular list ID.
example
sw-sb09-da(config)# access-list 1 permit 10.19.0.10 0.0.0.0 !sw-sb09-da(config)# access-list 1 permit 192.168.100.28 0.0.0.0 !sw-sb09-da(config)# access-list 2 permit 172.16.0.1 255.255.0.0
the resulting configs can be seen with show access-list:
!sw-sb09-da(config)# sh access-list access-list 1 10 permit host 10.19.0.10 access-list 1 20 permit host 192.168.100.28 access-list 2 10 permit 172.16.0.1 255.255.0.0
doing a "no access-list 1" will remove the first two entries of this ACL.
3.1.2. Configuring SNMP
Now that you have an ACL, you can specify which ACL entries may have access to SNMP services. This is done by specifying ACL entries by list ID in the snmp-server community context.
sw-sb09(config)# snmp-server community "NET" rw 1
here, "NET" is an arbitrary group name, rw means any member from the allowed group can read or modify SNMP parameters (as opposed to "ro", read-only), and "1" is the list ID of the group.
return to section index
return to main index
3.1.3. Accessing SNMP data
SNMP uses Management Information Bases (MIBs) to organize the data it can provide and modify on a device. SNMP data lookup is usually done by specifying the MIB values for a certain spec. The snmp context provides a set of tools for this - for a list of options, type "snmp ?".
example
here we do a system information query on the switch we are logged into:
sw-sb09# snmp walk 1.3.6.1.2.1.1 Name: sysDescr.0 Value: ALAXALA AX3640S AX-3640-48T2XW-L [AX3640S-48T2XW] Switching software Ver. 10.7 [OS-L3L] Name: sysObjectID.0 Value: ax3640s Name: sysUpTime.0 Value: 25724205 Name: sysContact.0 Value: Name: sysName.0 Value: sw-sb09 Name: sysLocation.0 Value: Name: sysServices.0 Value: 78
using snmpwalk
This is an utility for *nix machines, and can be used to prod the switch over the network, given you have the ACLs set up properly.
return to section index
return to main index
3.2 DHCP Relay Agent
A DHCP relay agent forwards DHCP messages between DHCP client and server residing on two different subnets and VLANs. This is a translation from the online NEC guide.
- configure client VLAN. This is the VLAN on which the DHCP clients live.
(config)# vlan 2 (config-vlan)# exit (config)# interface gigabitethernet 0/5 (config-if)# switchport mode access (config-if)# switchport access vlan 2 (config-if)# exit (config)# interface vlan 2 (config-if)# ip address 10.1.0.1 255.255.0.0 (config-if)# exit
- Next, set up the VLAN and interface connected to the DHCP server.
(config)# vlan 3 (config-vlan)# exit (config)# interface gigabitethernet 0/7 (config-if)# switchport mode access (config-if)# switchport access vlan 3 (config-if)# exit (config)# interface vlan 3 (config-if)# ip address 20.1.0.1 255.255.0.0 (config-if)# exit
- set the DHCP server's address as the helper address on client VLAN.
(config)# interface vlan 2 (config-if)# ip helper-address 20.1.0.10 (config-if)# exit
3.3 DHCP Server
The IP8800 comes with DHCP server capabilities. DHCP on the switch is provided on a VLAN basis, where the service is provided to hosts based on VLAN membership. DHCP configs for the basic DHCP server use the service dhcp and ip dhcp contexts.
- enable DHCP services for VLANs. Here, we want VLAN 1 to have the service.
(config)# service dhcp vlan 1
- specify address(es) excluded from address block. Here, IP addresses between 10.1 and 11.10 will not be handed out to clients.
!(config)# ip dhcp excluded-address 192.168.10.1 192.168.11.10
- configure client pool. Members of VLAN 1 will be part of pool "Test1", be given IP's from the 192.168.11.0/24 block, and will have 10-minute leases.
!(config)# ip dhcp pool Test1 !(dhcp-config)# network 192.168.11.0 255.255.255.0 !(dhcp-config)# lease 0 0 10
3.4 Port Mirroring
Say that you want to monitor the traffic to/from a certain port. The switch can act as a network tap if you configure port mirroring. When configured to port mirror, the switch would send a copy of the frames it receives from certain ports to another port where you'd, say, connect a network analyzer.
We needed to set up port mirroring to diagnose what was going wrong with the L2TP tunneling between the netFPGA hosts. The port you want to be the mirror port should not be associated with any VLANs, but be a plain old access port.
Syntax
monitor session <session no.> source interface gigabitethernet <src port> <tx|rx|both> destination interface gigabitethernet <mirror port>
- <session no.> - a number between 1-4 uniquely identifying the mirroring session. the switch can do a maximum of 4 sessions at once.
- <src port> - the port number(s) of the port(s) you're trying to monitor. Can be a range of ports e.g.
0/5-6,27. - <tx|rx|both> - mirror incoming, outgoing, or both traffics, respectively. "tx" and "rx" are with respect t the host connected to the port.
- <mirror port> - this is the port connected to the analyzer.
Errors
(config)# no monitor session 1 monitor : Can not delete it because data is not corresponding.
Rebooting the switch without reload stop sometimes corrupts the monitor configs. When this happens the monitor session will show up in your configurations but will not actually be there. Instantiating the same session again should fix things should this happen.
monitor : Mirror port and switchport are inconsistent.
This occurs if your monitor port is configured as an access port. remove any configs on the destination port to get rid of this error. Any changes made for this purpose should be placed in a arbitrary file (e.g. CHANGES - as in sw-top).
return to main index
3.5 Users and Passwords
adding a user
This is done with the adduser command as with any *nix machine (The new user is added to /etc/passwd as you expect). You need to be in enabled mode.
sw-sb09-da# adduser native no-flash
User(empty password) add done. Please setting password. <<---this means "user added. please set password".
Changing local password for native.
New password:
Retype new password:
!sw-sb09-da#
deleting a user
This is not done with deluser, but with rmuser.
setting/clearing passwords
password <username> will allow you to reset password. clear password <username> allows you to delete passwords. replacing <username> with "enable-mode" allows you to set the password for user admin (enabled mode).
Forgotten admin password
There is no recovery process akin to Cisco switches per se - pressing the reset button on the switch for 5 seconds will "default reset" the device. Once the switch reboots, you will be able to set new passwords - Default reset just clears the administration password, but will not wipe out your configurations (although the descriptions make it sounds like it does)
return to main index
3.6 DNS configuration
The IP8800 uses /etc/resolv.conf as with any other UNIX box. The file can be modified using the following command:
(config)# ip name-server <DNS server address>
side notes…IOS, disabling STP
Switch(config)#no spanning-tree vlan 1-1005
Switch(config)#exit
Switch#
1w4d: %SYS-5-CONFIG_I: Configured from console by console
Switch#sh spann
Switch#sh spanning-tree
No spanning tree instance exists.
IV OpenFlow Switching
This section goes over the OpenFlow aspects of the switch. This section mostly describes the old OpenFlow capable firmware, 11.1.C. The production version of the firmware, 11.1.Ae, no longer requires the SD card other than for the install. For updated flashing/operation instructions, refer to the link here.
4.1 What is OpenFlow?
OpenFlow is a virtualization technology developed by Stanford. Ideally, it allows administrators and researchers to configure the switch's behavior freely, without constraints of TCP/IP or the native firmware. This is achieved by removing traffic switching decisions off of the switch and onto an external, highly configurable controller. More on OpenFlow can be found at http://www.openflowswitch.org.
The switch supports OpenFlow in the form of virtual switches, or "datapaths" that overlay VLAN configurations; Each separate VLAN residing on the same device can be instantiated as a virtual switch that can be controlled by separate controllers and/or scripts, or be kept as a traditional VLAN that does commodity switching.
4.2 Outline
In order to make the switch OpenFlow capable, you need to flash the switch with an SD card containing specialized firmware. The rough outline of the steps are the following:
- disable Spanning Tree (STP)
- configure VLANs and ports, telnet
- copy firmware and license to SD card
- add openflow.conf to SD card
- reboot switch with SD card
- configure virtual switches
Since it'd be easier to describe what's going on, this process will be using the configurations for sw-sb-01 as an example. You should take a bit of time to graze over the configs for the switch before going on.
4.3 Configuring the switch
This section corresponds to steps 1 and 2 in the Outline.
1. Disable Spanning Tree (STP). STP compromises both OpenFlow and ORBIT nodehandler/nodeagent functionalities, so both STP and Per-VLAN Spanning Tree (PVST) must be shut down using the spanning-tree context from the (config)# prompt.
spanning-tree disable no spanning-tree vlan 1,3,11-14,19-26
The first line disables STP. The second line disables PVST for each VLAN used.
2. configure VLANs and ports, telnet. Because virtual switches are based on VLANs, all port and VLAN configs should be done before flashing. This part of the setup is just normal configuration using vlan, interface vlan, and interface gigabitethernet contexts.
4.4 Preparing the SD card
When flashing, the switch is booted off of an SD card containing the firmware, license, and a configuration file. Each switch should have come with an SD card; if not (or if the SD card is shoddy), it's ok to use another 128MB card. Anything larger may be affected by a known bug that causes filesystem corruption (see 'errors' section below ). This section is comprised of steps 3 and 4. Technically, it's better explained as three steps:
1. Copy license to SD card. The switch will refuse to boot from an SD card without the proper license file. Each license file should be named after a particular switch's serial ID, which can be found on a silver stcker on the lower righthand corner on the front of the device, or through the show ver command:
> sh ver
Date 2010/07/26 15:42:50 UTC
Model: AX3640S-48T2XW
S/W: OS-L3L Ver. 11.1.C
H/W: Main board
AX-3640-48T2XWE-L [WA13CL48T2XVS400E05A005:80330300:307:1B514-1B512]
Module slot1 PS-M(AC)
AX-F2430-PSA01 [WA0PSA010000C110005A005]
Module slot2 FAN-M
AX-F2430-FAN01 [WA2FAN010000C140005A005]
The string in square brackets right underneath "H/W: Main board" is the full serial number of the switch.
In this case, the proper license for this switch would be named 'WA13CL48T2XVS400E05A005.dat'. This should be renamed to 'license.dat' before being saved to the card.
2. Copy firmware image file to card. The firmware is named 'k.img'. This doesn't need to be renamed.
3. Create openflow.conf. Create a new file named 'openflow.conf' and save it to the card.
With the three files saved, the card is ready for use.
4.5 Flashing the switch
The switch has an SD slot next to the serial port. Slap the card in and reboot the switch with a reload as user admin.
sw-sb09-da# reload Dump information extracted? (y/n): y old dump file(rmdump 07/21 21:16) delete OK? (y/n): y
If connected by serial, you should be able to see the following startup sequence, which should take about 2 minutes:
sw-sb09-da# ROM 00.03.52 ....................................................................................... BOOT 00.03.00 Loading from dev2 100% login:
'Loading from dev2' indicates that the switch has successfully found, and is booting off of, the firmware on the SD card.
errors
If you forget the license, or use the wrong one (e.g. one whose name doesn't match the serial ID of the switch), The switch will restart in the ROM prompt.
The switch may also boot with the default image if the image on the SD card is corrupt. This is usually indicated by the message "Loading from dev0" and not recognizing OpenFlow specific commands.
Sometimes the switch will stop mounting the SD card without any apparent cause. The SD card is recognized as sdmem0a by the OS - in this case it will not show up if you do /sbin/mount. The apparent symptoms are:
- unrecognized OpenFlow commands
- nothing shows up under /mnt
If this was caused by someone accidentally removing the card, rebooting will fix this issue. If not, your best bet is to just prepare a new card.
The worst case scenario is a corrupted SD card. This may be caused by various reasons, including buggy firmware, improper restart, or using a card over 128MB. Symptoms include:
- the device not starting up
- the device loading the default image
- scrambled/truncated filenames (e.g. 'openflow.conf' showing up as 'openf~l.con')
- inability to create virtual switches
The most surefire way to fix this is to get a new card, or reformat the problematic one. The easiest way to do it is to use a Windows machine. When the card shows up under "Devices", right click and go to 'Format'. A window saying "Format Secure Digital Storage Device" should pop up. Set "Allocation Unit size" to "default allocation size", and click 'Start'. You may need to confirm that you want to reformat the card. The card should be good to go in about a minute or so.
Another known bug is a reported checksum error when booting from the SD card, despite having the proper license and an error-free kernel image. In this case the switch will boot from on-board flash (dev0):
ROM 00.03.52 ............................. Checksum error. ............................ BOOT 00.03.00 Loading from dev0 100% login:
This can allegedly be fixed by re-flashing the card, but I have yet to see it boot properly once the error shows up.
4.6 OpenFLow Commands
Index
these are the sub-sections to this section:
4.6.1. Background
4.6.2. Virtual switch instantiation
4.6.3. Virtual switch deletion
4.6.4. Manipulating virtual switches
4.6.5. Monitoring OpenFlow stats
4.6.6. other commands
The OpenFlow capable firmware supports a suite of commands that can be used to configure OpenFlow datapaths. These commands are located in /usr/local/bin/; try greping (is that a word?) for "vsi", or "port" for a list of commands.
Once the switch is booted up, we can move to the final step of actually instantiating virtual switches using these commands.
4.6.2. Virtual switch instantiation
setvsi
sw-sb-01's data VLAns are all OpenFlow virtual switches instantiated using the command setvsi. setvsi takes on the following minimal syntax:
setvsi <VLAN ID> <ports> tcp <IP addr:port> dpid <datapath ID>
- <VLAN ID> - the VLAN ID of the VLAN you want to overlay a virtual switch onto. Otherwise called the "VSI number"
- <ports> - the ports, without the '0/', that are going to be part of the virtual switch
- <IP addr:port> - IP address of the OpenFlow controller you want the virtual switch to listen to, and port number (default is 6633)
- <datapath ID> - 12-digit hex value defining the MAC address for the virtual switch. Always begins with '0x'
a full synopsis can be found by typing "setvsi" at the CLI.
setvsi basically echos the whole command into openflow.conf. This is handy since you can follow the exact commands used to instantiate the virtual switches on a particular switch by simply doing a cat /mnt/openflow.conf.
Some important things about VSI's…
- the VSI ID must match the VLAN ID.
- the list of ports must include all ports associated to that VLAN ID.
- VSI's cannot be instantiated on a native VLAN.
- If editing openflow.conf with a text editor, newline should be in UNIX convention (LF only).
implementation
Here's sw-sb-01's conf file:
sw-sb-01> cat /mnt/openflow.conf setvsi 12 31,32,48.12 tcp 172.16.0.4:6633 dpid 0x001010123132 setvsi 14 33,34,48.14 tcp 172.16.0.4:6633 dpid 0x001010143134 setvsi 20 35,36,48.20 tcp 172.16.0.4:6633 dpid 0x001010203230 setvsi 22 37,38,48.22 tcp 172.16.0.4:6633 dpid 0x001010223232 setvsi 24 39,40,48.24 tcp 172.16.0.4:6633 dpid 0x001010243234 setvsi 26 41,42,48.26 tcp 172.16.0.4:6633 dpid 0x001010263236
Each line is the exact command that was typed at the prompt. As you can see, sw-sb-01 has 6 virtual switches overlaying VLANs 12,14,20,22,24,and 26. All of them are pointed towards IP address 172.16.0.4, the ORBIT OpenFlow controller.
ORBIT DPID conventions
There are a few home-brew conventions that are used for the dpid's:
- first 3 bytes are always '001010'
- the 4th byte is the VLAN ID, in decimal
- the last two bytes are the VLAN ID in ascii hex notation. ASCII maps decimal 0-9 to hex 30-39.
So, for example, if we have VLAN 12, the last 3 bytes will be '12', '31', and '32'. That's the dpid from the first setvsi line in the terminal output above.
trunk ports
If you look at the list of ports from the conf file above, you'll notice that all of them end with "48.n", n being the VLAN ID for the virtual switch being instantiated. This value lets the switch know which VLAN ID to tag the traffic to/from a particular datapath with on a particular trunk port. For example, "48.12" means "tag virtual switch 12's traffic with VLAN ID 12 on trunk port 0/48." This is needed if you are using trunking the switch.
return to section index
return to main index
4.6.3. Virtual switch deletion
deletevsi deletes a certain virtual switch instantiation. The syntax is as follows:
deletevsi <VLAN ID>
nice and simple.
4.6.4. Manipulating virtual switches
You don't want to have to keep deleting and redoing virtual switches just to add or remove ports from an instantiation.
addport <VLAN ID> <port>- add ports to virtual switch for VLAN of <VLAN ID>deleteport <VLAN ID> <port>- delete ports from virtual switch for VLAN of <VLAN ID>
Nor do you want to keep setvsi-ing them because you didn't want a virtual switch one moment, and then you wanted it the next.
disablevsi <VLAN ID>- temporarily disable the virtual switch. As far as the switch is concerned, it's not there when it's disabled.enablevsi <VLAN ID>- enable a disabled virtual switch.
return to section index
return to main index
4.6.5. Monitoring OpenFlow stats
There are a few commands to pull stats about flows, virtual switches, ect.
showswitch <VLAN ID|cr> <detail|cr>- displays virtual switch information. VLAN ID and "detail" are optional parameters.showflow <VLAN ID|cr> <detail|cr>- display current flows. Same deal with parameters.showofinfo- display all OpenFlow and system (akin to doing a dmesg and sysctl -a back-to-back) stats
example
Displaying all datapaths:
sw-sb-01> showswitch vlan ports secure channel ---- ----- -------------- 12 31, 32, 48.12 connected 14 33, 34, 48.14 connected 20 35, 36, 48.20 connected 22 37, 38, 48.22 connected 24 39, 40, 48.24 connected 26 41, 42, 48.26 connected
Here you can see that all of the virtual switches are registered ("connected") with the controller. Now, if you want to focus on one virtual switch, say VLAN 12:
sw-sb-01> showswitch 12 detail
Virtual switch 12
Datapath ID : 68989104434(0x1010123132)
Datapath name: sw-sb-01
Port : gigabitethernet 0/31 (link down)
gigabitethernet 0/32 (link down)
gigabitethernet 0/48.12 (link up, 1G full)
Conn mode : tcp
Controller : 172.16.0.4:6633 (connected)
Exact match : 01-24, 49-50 hw 0 / hw max 3046
25-48 hw 0 / hw max 3046
Exact match : sw 0 / sw max 131072
Wildcard : sw 1 / sw max 100
Emerg : sw 0 / sw max 100
Packet buff : 256 packets / 4294967295 MB
Miss sendlen : 128 bytes
Wcard mode : S/W only
Counter type : Packet Counter
MAC learning : On
Min EQ : 0
Max EQ : 7
Default EQ : 4
VSI Enable : yes
4.6.6. other commands
I really haven't used the other commands e.g, for queue manipulation, so I can't really describe them. But here's the list of what I believe is all the commands for OpenFlow datapath manipulation:
-r-xr-xr-x 1 root wheel 221139 Jun 9 05:20 addport -r-xr-xr-x 1 root wheel 211566 Jun 9 05:20 deleteflow -r-xr-xr-x 1 root wheel 221162 Jun 9 05:20 deleteport -r-xr-xr-x 1 root wheel 202225 Jun 9 05:20 deletevsi -r-xr-xr-x 1 root wheel 202579 Jun 9 05:20 disablevsi -r-xr-xr-x 1 root wheel 202513 Jun 9 05:20 enablevsi -r-xr-xr-x 1 root wheel 521925 Jun 9 05:20 max-queue-rate -r-xr-xr-x 1 root wheel 436 Jun 9 05:20 ofconf_mng.sh -r-xr-xr-x 1 root wheel 202688 Jun 9 05:20 settrace -r-xr-xr-x 1 root wheel 231122 Jun 9 05:20 setvsi -r-xr-xr-x 1 root wheel 229570 Jun 9 05:20 showflow -r-xr-xr-x 1 root wheel 2146 Jun 9 05:20 showofinfo -r-xr-xr-x 1 root wheel 232831 Jun 9 05:20 showswitch
return to section index
return to main index
4.7. OF mode IP8800 and ORBIT
This section is meant for the odds-and-ends with respect to ORBIT deployments of the IP8800 in OpenFlow mode.
- SSH acess: user: native, passkey: geni101
- kvm-big: some of the VLANs on the IP8800 can be pointed to the Big Switch controller, at 172.16.0.14:6633. If doing so, don't forget to add the trunk as one of the virtual switch ports, e.g:
sw-sb-01> setvsi 22 37,38,48.22 tcp 172.16.0.14:6633 dpid 0x001010223232
This is actually the case with any OF switch, since it can only access controllers via the trunk. - SNAC: can be reached at 172.16.0.4. Port 6634 may be flowvisor.
- even when disabled with
disablevsi [vlan], virtual switches created on control VLANs will break Frisbee (e.g. imaging will time out). This is shown to be the case with all controllers present in ORBIT.
