Hurdle 3 (H3)

Objective

In this hurdle, teams will build a radio pair (transmitter and receiver) that can perform well in the presence of interference. The teams will be provided a baseline transmitter and receiver implementation. The transmitter will take input data from a specified source, packetize it and transmit it using a simple modulation scheme. The receiver module will take the received data and pipe it to a database file. Using these transmitter and receiver modules as starting points, the teams must write their own transmitter and receiver modules that can withstand a set of simple interference types and successfully deliver as many packets to the database over the course of 5 minutes.

By the end of the Qualification Period, teams must submit a final version of their radio pair for evaluation on ORBIT. In order to do this, teams must specify (in an email to challenge at orbit-lab dot org) the name of a saved image they wish to be tested. In the event that a team does not inform the Challenge of their saved image within 24 hours of the Qualification Period ending, then they will be evaluated using the originally provided baseline. The evaluation of each team's design will be performed after 24 hours following the completion of the Qualification Period.

Radio Operation: During development of their radio modules, teams will be rotated through the various USRP-enabled domains. Beyond ensuring fair exposure to ORBIT USRP resources, this allows teams to design/test their transmitter-receiver algorithms to work on as many different radios as possible. As in Hurdle 2, each domain has a specified 2.5MHz spectrum band that it is allowed to operate in. The ORBIT testbed monitors the frequencies used by each domain. Failure to restrict usage to the assigned frequency band may result in termination of the time slot.

The actual performance testing will take place on a pair of USRP N210 nodes that no contestant teams have been exposed to. Teams are expected to ensure that their software will operate and adapt accordingly to slight differences in hardware performance (e.g. device sensitivities). Further, the precise 2.5MHz band that will be used for evaluation will not be announced, though the teams are advised that the evaluation will occur between 2.1GHz and 3GHz. Teams also should note that none of the USRP-enabled domains have been assigned to operate in this band. Thus teams must ensure that their software can work equally well on an arbitrary 2.5MHz band without being given the precise band they will be tested on.

Interference Scenario: The submitted radios will be evaluated against a choreographed "interference scenario" that will be built using a random time sequence of the following interference types:

N0 = one second period of no interference N1 = one second period of short-term 1.25MHz band-limited white noise interference signal that resides in the lower half of the 2.5MHz band N2 = one second of short-term 1.25MHz band-limited white noise interference signal that resides in the upper half of the 2.5MHz band.

The interference in each one second interval of the "interference scenario" is independent and equally likely to be any of the three interference types. As an example, the interference scenario may be represented as the sequence:

N0, N2, N2, N1, N0, N1, N2,…

Teams will not be provided the specific choreographed interference scenario that will be used for testing, but instead must be prepared for any random sequence of interference types N0, N1, and N2. During periods of interference, the interference powers will be set at a level such that the received SNR might range from 0dB SNR to 10dB SNR.

The actual testing interference scenario will be restricted to an unspecified 2.5MHz of bandwidth running on USRPs that no teams have been exposed to. Teams are reminded that during development of their modules they must operate their radios (and any interference testing) within the spectrum band assigned to their domain Teams are advised that to conduct their own testing of interference they should use the two USRP's provided with each virtual domain. To study interference, teams may either introduce synthetic interference directly at the receiver, or may use the transmitter to inject both legitimate communications and interference. Teams must not transmit at frequencies outside of their assigned domain range.

Evaluation Criteria: Teams will be evaluated on the total number of correct packets received and delivered to the database during the 5 minutes of evaluation time. Teams are provided Layer-2 packets and are allowed to make their own PHY frames; it is only the Layer-2 packets that get evaluated by the receiver. Teams are reminded that during the implementation of their own radios they MUST use the data input and output methods provided in the baseline radio.

Instructions

WINLAB has created 16 virtual domains containing USRP2s and USRP N210s. Given the large number of teams participating in the Challenge Hurdles, teams will be assigned specific times at which they may access specified virtualdomains containing USRP2 and USRP N210s. In order to ensure fairness, teams will rotate through different domains, and hence no team will have a significantly larger amount of time on a particular domain compared to other teams. At the end of each 2 hour time period, teams will be removed from their domain in order to make that domain available for the next team. Teams are advised to plan their development and work according to the schedule slots they have been assigned. No reassignment or additional time slots will be given to any team.

a.) Look up your team's assigned time slot for a USRP virtual domain at ​scheduler page

All teams have been given their time assignments, with domain name and frequency band, in an initial email from the Challenge. Note that each domain has a corresponding frequency band that must be used with that domain. The allowed frequency band can also be found using the online schedule by running the cursor over the virtual domain's name.

b.) ssh into console: Teams may access their domain at the corresponding assigned timeslot by:

  ssh username@console.grid.orbit-lab.org

Note that if a team does not use its time slot, then it will not be able to obtain a replacement time slot. Time slots have been assigned so as to give each team a fair and equitable exposure to USRP resources on ORBIT. No requests for additional time slots will be approved.

c.) Image nodes with a specified image:

  omf load -i darpahurdle23.ndz -t system:topo:all

The execution and completion of this command may take several minutes as software is being installed onto the nodes.

d.) Modify the benchmark_tx3 and benchmark_rx3 radio modules provided on the image. Teams are reminded that their objective in Hurdle 3 is to create a transmitter and receiver resilient to the interference scenario described above. The provided benchmark_tx3 pulls packetized data from the ORBIT server (which also records every packet into the OML database) and transmits it over the air; benchmark_rx3 receives packets and sends correctly received packets to the OML server.

NOTE: Evaluation will be performed by running the team's transmitter with the name benchmark_tx3 and receiver with the name benchmark_rx3. Teams must submit their final modules with these names. Teams are not permitted to change any FPGA code running on the USRP radios. Operating bandwidth must be limited to 2.5 MHz. The ORBIT testbed monitors the frequencies used by each domain. Failure to restrict usage to the assigned frequency band may result in termination of the time slot. Since teams will have a limited exposure to ORBIT for development each day, teams must remember to save their image before the end of their time slot. Also, please note that in order to be evaluated for Hurdle 3, teams must store their modified image and register it as final. This (single) image must contain both the modified benchmark_tx3 and benchmark_rx3.

For instructions on preparing and saving an image, teams should read how to save an image instructions.

In order to save their modified, teams should first issue the prepare command:

  ./prepare.sh

This will remove udev rules (to prevent renaming of interfaces) and dump log files to lower the size of the image. It also shuts down the node. Next, teams should then issue the following command:

  omf save -n nodename.grid.orbit-lab.org

This will save the node image into the image repository with the unique name displayed on the screen.

Teams are reminded that it is prudent to give themselves enough time to save their work before the end of their time slot. After saving their final image, teams must email the Challenge at challenge at orbit-lab dot org to inform the Challenge of the team's final submission. This email must contain the name of the saved image (as provided on the screen after issuing the omf save command) that the team wishes to submit for evaluation. Teams that do not inform the Challenge of their saved image within 24 hours of the Qualification Period ending will be evaluated using the originally provided baseline. Teams should also note that the names of their radio transmitter and receiver must be benchmark_tx3 and benchmark_rx3, respectively,with unmodified packet source and packet destination modules, and command line argument for center frequency specification. Evaluation will only be performed using benchmark_tx3 and benchmark_rx3 names.

The Qualification Period ends March 5 at 11:59:59PM Eastern Standard Time.

Validation

Teams will be ranked according to the number of error free packets they successfully transmit during a 5 minute evaluation run. The evaluation run will consist of running the interference scenario, then initiating the team's receiver, then initiating the team's transmitter. In the unlikely event that teams end up with the same number of error free packets, the shortest transfer time will be used to break ties. A selection of the top performing teams will be chosen as contestants for the DARPA Challenge. Teams not selected may still be able to participate in the DARPA Spectrum Challenge as "Wild Card Teams." Further information related to Wild Card selection will be made available following the completion of the hurdles.

Questions or issues related to the hurdles should be sent by email to dsc-questions at orbit-lab dot org. Answers will be posted to a public forum site at Q&A page.

Teams are reminded that the Qualification Period completes on March 5, 2013 at 23:59:59pm (Eastern). Submissions for Hurdle 3 must be turned in at the end of your time slot on that day.

We again congratulate you in making it to Hurdle 3, and wish you the best of luck in designing your radio.

—ORBIT Spectrum Challenge Team