Changes between Version 47 and Version 48 of Other/Summer/2025/self-driving


Ignore:
Timestamp:
Jul 17, 2025, 11:53:49 PM (9 days ago)
Author:
MichelleGutwein
Comment:

Legend:

Unmodified
Added
Removed
Modified
  • Other/Summer/2025/self-driving

    v47 v48  
    5151[https://docs.google.com/presentation/d/1cWWVyynt3JQVEMevvJNloXy6zKUr8TI4Zk4XfLalfAs/edit?slide=id.p#slide=id.p Week 7 slides]//
    5252
     53== Week 8
     54Tested the car to see if it would stop at intersections. It didn't. [[BR]]
     55Found out that in addition to automatically labeling lookahead point angles, we also needed to manually label intersection frames from the data [[BR]]
     56Now the car stops at most intersections [[BR]]
     57Recorded more data at and around the main intersection to fine tune this [[BR]]
     58Started work on creating a mapping system of the intersection area by measuring the length and width of the lanes [[BR]]
     59\\
     60[https://docs.google.com/presentation/d/1qi_ZzB_SwP99b-R7k28fyWV2QZ1rjOJGeZy_W1YOxq4/edit?slide=id.p#slide=id.p Week 8 slides]//
    5361
    5462== Connecting to Rascal
     63
     64Through Zerotier: [[BR]]
     651. Sign into Zerotier under the account **rascalstonesdc-at-gmail.com**   password: **si2022sdc** [[BR]]
     662. Connect to the network: rascalstonesdc's 1st network **(ID: d3ecf5726d1a9fcd)** [[BR]]
     673. Now you can open a terminal and connect to the IP address of the rascal using the command: **ssh rascal@10.244.114.238 [[BR]]**
     684. Next, run the command: **roslaunch rascal pure_pursuit.launch** to launch all the nodes [[BR]]
     695. In a web page type **10.244.114.238:5000** to open the web display where you can record data [[BR]]
     706. In a separate terminal, enter **ssh rascaltrain@10.244.176.240** to connect to the rascal server [[BR]]
     717. If you get an error message that says: **access denied public key** then connect through ORBIT (instructions below) to add your public key to the /.ssh file titled **authorized_keys** [[BR]]
     728. Now you should be able to connect to the server through Zerotier [[BR]]
     73
    5574Through ORBIT: [[BR]]
    56751. Make a reservation on your orbit account under the **intersection** bed [[BR]]
    57762. Open a new terminal and type **ssh <your username>@console.intersection.orbit-lab.org** [[BR]]
    58773. You should see the 'welcome to ORBIT-LAB' sign [[BR]]
    59 4. Enter **rascal@<ip address to come>** for the rascal
    60 5. In a separate terminal, enter **ssh rascaltrain@srv1** for the rascal server [[BR]]
    61 
    62 [[BR]]
    63 
    64 Through Zerotier: [[BR]]
    65 1. Sign into Zerotier under the account **rascalstonesdc-at-gmail.com**   password: **si2022sdc**[[BR]]
    66 2. Connect to the network: rascalstonesdc's 1st network **(ID: d3ecf5726d1a9fcd)**[[BR]]
    67 3. Now you can open a terminal and connect to the IP address of the rascal using the command: **ssh rascal@10.244.114.238[[BR]]**
    68 4. Next, run the command: **roslaunch rascal pure_pursuit.launch** to launch all the nodes[[BR]]
    69 5. In a web page type **10.244.114.238:5000** to open the web display where you can record and edit data for training[[BR]]
    70 6. In a separate terminal, enter **ssh rascaltrain@10.244.176.240** to connect to the rascal server [[BR]]
     784. Enter **ssh rascaltrain@srv1** for the rascal server [[BR]]
    7179
    7280== Collecting Data
     
    75832. Use the joystick or keys on the keyboard (wasd) to move the car around[[BR]]
    76843. Press **bagrecorder** again to stop recording [[BR]]
    77 4. Open data **opt/upcar/dataProcessing** and type **python bag2AllData.py** in the terminal to access all recorded sessions[[BR]]
    78 5. Select the number that you want to download (should be the most recent one)[[BR]]
    79 6. After it finishes downloading you can type the command: **roslaunch rascal sim.launch** which will upload that session to the web display[[BR]]
    80 7. In the web display you can play the video to see what the camera recorded (you might need to refresh the page to see the session come up)[[BR]]
    81 8. You can now edit the data that will be used for training[[BR]]
    82 9. NOTE: for the discrete waypoint model you MUST label the data manually at each segment. Highlight the portion of data that has multiple valid paths (left, right, and straight) then click **replayer/setLvalid, replayer/setRvalid, replayer/setSvalid** to indicate which paths are possible at that intersection. This will be used later when training.[[BR]]
    83 10. To save data, run the **data_loader/smoothen** command
     854. You can record multiple segments in a loaded session [[BR]]
     865. When finished, press ctrl+c in the terminal to save the recordings [[BR]]
     876. Go to **/opt/upcar/dataProcessing** and type **python bag2AllData.py** in the terminal to access all recorded sessions [[BR]]
     887. Select the number that you want to download (should be the most recent one)[[BR]]
     898. Open a new terminal and connect to the rascal server [[BR]]
     909. Open **/upcar/dataProcessing** on the server and type **./copyFromCar.sh** which will get the files from the car and bring them over to the server [[BR]]
     9110. It will ask you to select a host. Choose **1) rascal@172.24.114.238** which is the car's IP address [[BR]]
     9211. If it asks for a password it's always **si2022sdc** [[BR]]
     9312. It will ask you which sessions you want to copy over (enter the indices of the sessions you want to copy) [[BR]]
    8494
    85 == Deleting Bad Data Segments
     95== Editing and labeling the Data
     961. In the server's terminal, type the command: **roslaunch rascal sim.launch** which will launch the web dislpay for the server where you can edit the desired session [[BR]]
     972. In a web page type **10.244.176.240:5000** to open the web display where you can edit data for training (it's best to use the server's web display instead of the car's) [[BR]]
     983. In the web display you can click **replayer/pause-play** to run through all of the data segments [[BR]]
     99
     100**Deleting bad segments of data:**
    86101When portions of the recorded data are messy or irrelevant for training, they can be excluded to improve model performance [[BR]]
    87 1. In the web display you can click **replayer/pause-play** to run through all of the data segments [[BR]]
    88 2. To jump between different segments that the web display automatically divides the data into, click **replayer/nextSeg** or **replayer/prevSeg**
    89 3. When you get to a segment you want to delete, pause the run through and hit **replayer/editMode** [[BR]]
    90 4. Select the data you want to delete by using the **replayer/jumpEnd** and **replayer/jumpStart** buttons [[BR]]
    91 5. Select **replayer/editMode** again to highlight the part that's going to be deleted [[BR]]
    92 6. Click **replayer/deleteSelection**  to delete the segment [[BR]]
     1021. To jump between different segments that the web display automatically divides the data into, click **replayer/nextSeg** or **replayer/prevSeg**
     1032. When you get to a segment you want to delete, click the desired starting point on the orange path and hit **replayer/editMode** [[BR]]
     1043. Select the endpoint of the segment you want to delete by clicking the orange path at that point [[BR]]
     1054. Select **replayer/editMode** again to highlight the part that's going to be deleted [[BR]]
     106   -  You can use the **replayer/jumpStart** and **replayer/jumpEnd** buttons to go to the start or end of a segment [[BR]]
     107   -  You can use the **replayer/forward** and **replayer/backward** to go back and forth between frames in a given segment to help get exact data highlighted [[BR]]
     1085. Click **replayer/deleteSelection** to delete the highlighted segment [[BR]]
     109
     110**Labeling intersection data:** [[BR]]
     111For intersection data you MUST label the data manually at each segment where an intersection occurs [[BR]]
     1121. Highlight the frames of data that has multiple valid paths (left, right, and straight) the same way you would for deleting segments [[BR]]
     1132. Click **replayer/setLvalid, replayer/setRvalid, replayer/setSvalid** to indicate which paths are possible at that intersection. This will be used later when training.[[BR]]
     114
     115Once you are finished editing you MUST save the data by running the **data_loader/smoothen** command
    93116
    94117== Training Rascal
    95118We used the fisheye and discrete_waypoint model to train on [[BR]]
    96 1. Open a new terminal and connect to the rascal server's IP address by using this command: **ssh rascaltrain@10.244.176.240** (or use ORBIT) [[BR]]
    97 2. Open **/upcar/dataProcessing** and type **./copyFromCar.sh** which will get the files from the car and bring them over to the server [[BR]]
    98 3. It will ask you to select a host. Choose **1) rascal@172.24.114.238** which is the car's IP address [[BR]]
    99 4. If it asks for a password it's always **si2022sdc** [[BR]]
    100 5. It will ask you which sessions you want to copy over (pick the session version of the files you want that says smooth) [[BR]]
    101 6. If you go to **/sessions** and type **ls** your chosen files should be listed there [[BR]]
    102 7. Go back to **/upcar** and move your data to /rascaltraining/data: **mv dataProcessing/sessions/<your file name here> rascaltraining/data [[BR]]
    103 8. If you go into **/rascaltraining/data** you should see your file listed [[BR]]
    104 9. Go back to **/rascaltraining** and type the command: **python trainModel.py -m fisheye --use_labels 1.0 -s labels** when using fisheye. Type the command:  **python trainModel.py -m discrete_waypoint --center_width 30 --num_bins 3 --use_labels 1.0 -s labels** when using discrete_waypoint[[BR]]
    105 10. Now your model is being trained [[BR]]
    106 11. To stop training press enter [[BR]]
    107 12. Remember to stop training when the validation loss doesn't seem to be decreasing by a significant amount anymore to avoid overfitting [[BR]]
    108 13. To check what number model you just created, go to **/savedModels** and type **ls** (yours will be the most recent number)
    109 14. To push your newly made model to gitlab, type **git add -f <your model number here>** [[BR]]
    110 15. Next type the command **git commit -m "<give it a name>"** [[BR]]
    111 16. Next type the command **git push origin main** [[BR]]
    112 17. Finally, back in the car's terminal go to **/savedModels** and type **git pull origin main** [[BR]]
     1191. If you go to **/sessions** in the server terminal and type **ls** your chosen files should be listed there [[BR]]
     1202. Go back to **/upcar** and move your data to /rascaltraining/data: **mv dataProcessing/sessions/<your file name here> rascaltraining/data [[BR]]
     1213. If you go into **/rascaltraining/data** you should see your file listed [[BR]]
     1224. Go back to **/rascaltraining** and type the command: **python trainModel.py -m fisheye --use_labels 1.0 -s labels** when using fisheye. Type the command:  **python trainModel.py -m discrete_waypoint --center_width 30 --num_bins 3 --use_labels 1.0 -s labels** when using discrete_waypoint[[BR]]
     1235. Now your model is being trained [[BR]]
     1246. To stop training press enter [[BR]]
     1257. Remember to stop training when the validation loss doesn't seem to be decreasing by a significant amount anymore to avoid overfitting [[BR]]
     1268. To check what number model you just created, go to **/savedModels** and type **ls** (yours will be the most recent number)
     1279. To push your newly made model to gitlab, type **git add -f <your model number here>** [[BR]]
     12810. Next type the command **git commit -m "<give it a name>"** [[BR]]
     12911. Next type the command **git push origin main** [[BR]]
     13012. Finally, back in the car's terminal go to **/savedModels** and type **git pull origin main** [[BR]]
    113131
    114132You're now ready to start testing! [[BR]]