Changes between Version 18 and Version 19 of Other/Summer/2023/RobotTestbed
- Timestamp:
- Jul 20, 2023, 3:29:46 PM (16 months ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
Other/Summer/2023/RobotTestbed
v18 v19 17 17 18 18 19 The second goal is automating the labeling process of the activity within the environment using the natural language descriptions of the data provided in video format. The video auto-labeling can be done using neural networks (ex: CNN and LSTM) in an encoder-decoder architecture for both feature extraction and the language model. For activities that cannot be classified within a specific amount of certainty, the auto-labeling tool could save the time stamp and/or video clip and notify the user that it requires manual labeling. In the case the network is fully trained, it would simply choose the label with the highest probability and possibly mark that data as “uncertain”. The main goal is to connect this video data to the sensor data in hopes to bridge the gap between sensor-to-text.19 The second goal is automating the labeling process of the activity within the environment using the natural language descriptions of video data. The video auto-labeling can be done training neural networks (ex: CNN and LSTM) in an encoder-decoder architecture for both feature extraction and the language model. For activities that cannot be classified within a specific amount of certainty, the auto-labeling tool could save the time stamp and/or video clip and notify the user that it requires manual labeling. In the case the network is fully trained, it would simply choose the label with the highest probability and possibly mark that data as “uncertain”. The main goal is to connect this video data to the sensor data in hopes to bridge the gap between sensor-to-text. 20 20 21 === The Projects's Three Phases === 22 '''Scenario One''' 23 24 '''Scenario Two''' 25 26 '''Scenario Three''' 21 27 22 28 == Progress Overview ==