Changes between Initial Version and Version 1 of Tutorials/i1Net/mgen

Timestamp:

Jun 11, 2018, 4:17:31 PM (6 years ago)

Author:

nilanjan

Comment:

—

Tutorials/i1Net/mgen

@@ +1 @@
+== Multi-Genenrator with real-time data logging ==
+
+Multi-Generator (mgen) is an open source network utility capable of performing performance tests and measurements by generating and receiving UDP/TCP packets. This tool is already well documented under the [https://en.wikipedia.org/wiki/Wikipedia:Tutorial/Wikipedia_links Networks and Communication Systems Branch at US Navy Research]. 
+
+An integrated version of mgen (version 5.02c) and Orbit Measurement Framework & Library (OML) has been built and tested in the Orbit environment. The following steps describe how to build and run this version of mgen.
+
+
+=== Set up ===
+
+1) Install the OML library and develpoment files.
+{{{
+apt-get update
+apt-get install liboml2
+}}}
+
+While building mgen other dependencies may be required to complete the compilation. Simply search and install the missing libraries and continue compiling.
+
+
+2) Download the source file for mgen, navigate to 'makefiles' directory and build.
+{{{
+wget https://downloads.pf.itd.nrl.navy.mil/mgen/src-mgen-5.02c.tgz
+tar -zxvf src-mgen-5.02c.tgz
+mv src-mgen-5.02c.tgz mgen
+cd mgen/makefile
+make -f Makefile.linux mgen
+}}}
+
+3) use help for a listing of options
+{{{
+./mgen help
+GPSSubscribe(): fopen() error: No such file or directory
+GPSSubscribe(): fopen() error: No such file or directory
+mgen: version 5.02c
+mgen [ipv4][ipv6][input <scriptFile>][save <saveFile>]
+     [output <logFile>][log <logFile>][hostAddr {on|off}
+     [logData {on|off}][logGpsData {on|off}]
+     [binary][txlog][nolog][flush]
+     [event "<mgen event>"][port <recvPortList>]
+     [instance <name>][command <cmdInput>]
+     [sink <sinkFile>][block][source <sourceFile>]
+     [interface <interfaceName>][ttl <multicastTimeToLive>]
+     [unicast_ttl <unicastTimeToLive>]
+     [df <on|off>]
+     [tos <typeOfService>][label <value>]
+     [txbuffer <txSocketBufferSize>][rxbuffer <rxSocketBufferSize>]
+     [start <hr:min:sec>[GMT]][offset <sec>]
+     [precise {on|off}][ifinfo <ifName>]
+     [txcheck][rxcheck][check]
+     [queue <queueSize>][broadcast {on|off}]
+     [convert <binaryLog>][debug <debugLevel>]
+     [gpskey <gpsSharedMemoryLocation>]
+     [boost] [reuse {on|off}]
+     [oml <server:port[,omlFile]>]
+MgenApp::ProcessCommands() OnCommand(help) error
+mgen: error while processing startup commands
+}}}
+
+The errors shown above are just informatory and can be ignored.
+
+All the options above (with the exception of 'oml') are explained in details in the [https://downloads.pf.itd.nrl.navy.mil/docs/mgen/mgen.html reference guide].
+
+The 'oml' option specifies the oml server and port number to send and record events into an sql file. A specific filename can be given as well otherwise if the field is left blank the filename will default to 'oml_mgen_test.sq3'. Example
+{{{
+./mgen event "listen udp 5124" oml oml:3003,test_file
+}}}
+
+The table below describes the parameters that are recorded
+
+ || '''Parameter''' || '''Type'''    || '''Description''' ||
+ || oml_sender_id || INTEGER || identify which node the recorded data came from || 
+ || oml_seq       || INTEGER || oml sequence number ||
+ || oml_ts_client || REAL    || oml client time stamp ||
+ || oml_ts_server || REAL    || oml server time stamp ||
+ || "proto"       || TEXT    || type of traffic -> UDP, TCP ||
+ || "direction"   || TEXT    || direction of traffic -> sender, receiver ||
+ || "flowid"      || INTEGER || mgen flow id ||
+ || "seq_num"     || INTEGER || sequence number for each flow id ||
+ || "src_addr"    || TEXT    || source IP address ||
+ || "src_port"    || INTEGER || source port ||
+ || "dst_addr"    || TEXT    || destination IP ||
+ || "dst_port"    || INTEGER || destination port ||
+ || "data_len"    || INTEGER || length of data ||
+
+=== Simple OML recording with 1 sender and 1 receiver ===
+1) Refer to the reference guide for complete details on using formatted scripts and options to run mgen. In this section we'll only describe 
+
+2) To demonstrate the OML recording feature, setup mgen on two nodes and run a script with the input option. For this example node6-6 and node15-6 on the grid were used and the corresponding script is showing the table.
+ || '''Node Id''' || '''Traffic Direction''' || '''Script contents''' ||
+ || node6-6 || sender || 0.0 ON 0 UDP SRC 5000 DST 10.10.15.6/5000 POISSON [10 1024:1200] \\ 2.0 ON 1 UDP SRC 5001 DST 10.10.15.6/5001 POISSON [8 1024:1200] \\ 4.0 ON 2 UDP SRC 5002 DST 10.10.15.6/5002 POISSON [6 1024:1200] \\ 6.0 ON 3 UDP SRC 5003 DST 10.10.15.6/5003 POISSON [4 1024:1200] \\ 8.0 ON 4 UDP SRC 5004 DST 10.10.15.6/5004 PERIODIC [1 256] ||
+ || node15-6 || receiver || 0.0 LISTEN UDP 5000-5004 || 
+
+
+The sender script file (udp_sender_script.mgn) configures the node to transmit UDP traffic to the receiver node which is configured to listen for UDP packet on port ranging from 5000 to 5004.
+
+The format of the sender script is briefly described below.
+ || Event time || Event || Flow id || Protocol || Source port || Destination IP and port || Traffic pattern ||
+ || 0.0 || ON || 0 || UDP || SRC 5000 || DST 10.10.15.6/5000 || POISSON [10 1024:1200]||
+
+Likewise the receiver script (udp_recv_script.mgn) is condensed into a single liner 
+ || Event time || Event  || Protocol || Port list ||
+ || 0.0        || LISTEN || UDP      || 5000-5004 ||
+
+3) Run each script using the 'input' option of mgen and specify the oml options for recording. 
+{{{
+node6-6> ./mgen input udp_send_script.mgn       oml oml:3003,udp_send_mgen
+
+node15-6> ./mgen input udp_recv_script.mgn oml oml:3003,udp_recv_mgen
+}}}
+
+Each instance of mgen will start producing the text output. In the background it will also send packet infomation to the specified oml server and database. To quickly view the contents of the data files, run the following command from the console:
+{{{
+console> sqlite3 /var/lib/oml2/udp_recv_mgen.sq3 .dump
+PRAGMA foreign_keys=OFF;
+BEGIN TRANSACTION;
+CREATE TABLE _senders (name TEXT PRIMARY KEY, id INTEGER UNIQUE);
+INSERT INTO "_senders" VALUES('et',1);
+CREATE TABLE "_experiment_metadata" (oml_tuple_id INTEGER PRIMARY KEY, oml_sender_id INTEGER, oml_seq INTEGER, oml_ts_client REAL, oml_ts_server REAL, "subject" TEXT, "key" TEXT, "value" TEXT);
+INSERT INTO "_experiment_metadata" VALUES(1,NULL,NULL,NULL,NULL,NULL,'table__experiment_metadata','0 _experiment_metadata subject:string key:string value:string');
+INSERT INTO "_experiment_metadata" VALUES(2,NULL,NULL,NULL,NULL,NULL,'start_time','1528726472');
+INSERT INTO "_experiment_metadata" VALUES(3,NULL,NULL,NULL,NULL,NULL,'table__mp__udp_recv_mgen','1 _mp__udp_recv_mgen proto:string flowid:int32 seq_num:int32 src_addr:string src_port:int32 dst_addr:string dst_port:int32 data_len:int32');
+CREATE TABLE "_mp__udp_recv_mgen" (oml_tuple_id INTEGER PRIMARY KEY, oml_sender_id INTEGER, oml_seq INTEGER, oml_ts_client REAL, oml_ts_server REAL, "proto" TEXT, "flowid" INTEGER, "seq_num" INTEGER, "src_addr" TEXT, "src_port" INTEGER, "dst_addr" TEXT, "dst_port" INTEGER, "data_len" INTEGER);
+INSERT INTO "_mp__udp_recv_mgen" VALUES(1,1,1,3.50581999868154525756e-01,2.95809999999999961861e-01,'UDP',0,4,'10.10.6.6',5000,'10.10.15.6',5000,1044);
+INSERT INTO "_mp__udp_recv_mgen" VALUES(2,1,2,4.55220999661833047866e-01,0.3979,'UDP',0,5,'10.10.6.6',5000,'10.10.15.6',5000,1172);
+INSERT INTO "_mp__udp_recv_mgen" VALUES(3,1,3,0.585224999114871,5.2790399999999992886e-01,'UDP',0,6,'10.10.6.6',5000,'10.10.15.6',5000,1192);
+INSERT INTO "_mp__udp_recv_mgen" VALUES(4,1,4,1.40533899888396263122e+00,1.489031,'UDP',0,7,'10.10.6.6',5000,'10.10.15.6',5000,1041);
+INSERT INTO "_mp__udp_recv_mgen" VALUES(5,1,5,1.59353199973702430725e+00,1.614587,'UDP',0,8,'10.10.6.6',5000,'10.10.15.6',5000,1110);
+INSERT INTO "_mp__udp_recv_mgen" VALUES(6,1,6,1.68116699904203414916e+00,1.62387099999999984234e+00,'UDP',0,9,'10.10.6.6',5000,'10.10.15.6',5000,1198);
+INSERT INTO "_mp__udp_recv_mgen" VALUES(7,1,7,1.72220599837601184844e+00,1.66488099999999983324e+00,'UDP',0,10,'10.10.6.6',5000,'10.10.15.6',5000,1176);
+INSERT INTO "_mp__udp_recv_mgen" VALUES(8,1,8,1.89545799978077411651e+00,1.838213,'UDP',1,0,'10.10.6.6',5001,'10.10.15.6',5001,1049);
+INSERT INTO "_mp__udp_recv_mgen" VALUES(9,1,9,1.91608599945902824401e+00,1.858813,'UDP',1,1,'10.10.6.6',5001,'10.10.15.6',5001,1127);
+INSERT INTO "_mp__udp_recv_mgen" VALUES(10,1,10,2.05955099686980247497e+00,2.144107,'UDP',1,2,'10.10.6.6',5001,'10.10.15.6',5001,1169);
+INSERT INTO "_mp__udp_recv_mgen" VALUES(11,1,11,2.0766779966652393341e+00,2.269656,'UDP',1,3,'10.10.6.6',5001,'10.10.15.6',5001,1080);
+INSERT INTO "_mp__udp_recv_mgen" VALUES(12,1,12,2.10336599871516227722e+00,2.269673,'UDP',1,4,'10.10.6.6',5001,'10.10.15.6',5001,1064);
+INSERT INTO "_mp__udp_recv_mgen" VALUES(13,1,13,2.15250899642705917358e+00,2.269681,'UDP',1,5,'10.10.6.6',5001,'10.10.15.6',5001,1075);
+INSERT INTO "_mp__udp_recv_mgen" VALUES(14,1,14,2.19350999966263771057e+00,2.269688,'UDP',0,11,'10.10.6.6',5000,'10.10.15.6',5000,1180);
+INSERT INTO "_mp__udp_recv_mgen" VALUES(15,1,15,2.24396899715065956115e+00,2.269796,'UDP',1,6,'10.10.6.6',5001,'10.10.15.6',5001,1073);
+INSERT INTO "_mp__udp_recv_mgen" VALUES(16,1,16,2.24748399853706359863e+00,2.269805,'UDP',0,12,'10.10.6.6',5000,'10.10.15.6',5000,1161);
+INSERT INTO "_mp__udp_recv_mgen" VALUES(17,1,17,2.26098299771547317504e+00,2.269826,'UDP',1,7,'10.10.6.6',5001,'10.10.15.6',5001,1148);
+INSERT INTO "_mp__udp_recv_mgen" VALUES(18,1,18,2.2951769977808,2.269834,'UDP',0,13,'10.10.6.6',5000,'10.10.15.6',5000,1164);
+INSERT INTO "_mp__udp_recv_mgen" VALUES(19,1,19,2.30865999683737754821e+00,2.26984,'UDP',1,8,'10.10.6.6',5001,'10.10.15.6',5001,1128);
+INSERT INTO "_mp__udp_recv_mgen" VALUES(20,1,20,2.32708599790930747985e+00,2.269847,'UDP',1,9,'10.10.6.6',5001,'10.10.15.6',5001,1171);
+INSERT INTO "_mp__udp_recv_mgen" VALUES(21,1,21,2.35045599937438964843e+00,2.293127,'UDP',0,14,'10.10.6.6',5000,'10.10.15.6',5000,1153);
+INSERT INTO "_mp__udp_recv_mgen" VALUES(22,1,22,2.44623499736189842224e+00,2.388898,'UDP',0,15,'10.10.6.6',5000,'10.10.15.6',5000,1166);
+INSERT INTO "_mp__udp_recv_mgen" VALUES(23,1,23,2.58714799955487251281e+00,2.52982499999999976836e+00,'UDP',1,10,'10.10.6.6',5001,'10.10.15.6',5001,1173);
+
+}}}
+