====== LLM-Based VR Activity Recognition ======

This project aims to recognize human activities through analyzing sensor data with artificial intelligence (AI) technologies. 

Collecting sensor data from VR headsets & controllers and develop machine learning models to classify human activities

This project will also involve leveraging large language models (LLMs) for sensor data interpretation.

=== Week 1: [] 

Studied paper "Real-Time Recognition of In-Place Body Actions and Head Gestures using Only a Head-Mounted Display"

Studied paper "FitCoach: Virtual fitness coach empowered by wearable mobile devices"

=== Week 2: [] 
Started setting up necessary programs on our devices 
 - Android Studio, Arctic Fox
 - MATLAB

Worked on slides template to use for future presentations.


=== Week 3: [[https://docs.google.com/presentation/d/10RiVvwob7MiTjVPdoz9zFv8Z5Y2O55E6WgtBuDEMQMU/edit?slide=id.g3704141428e_2_180#slide=id.g3704141428e_2_180|Slides]] 

Studying papers related to activity recognition using motion sensor 
 - Motion sensor from VR: Real-Time Recognition of In-Place Body Actions and Head Gestures using Only a Head-Mounted Display [1]
 - Motion sensor from wearable devices: FitCoach: Virtual fitness coach empowered by wearable mobile devices [2]

Studying the concept of Inertial Measurement Unit (IMU)
 - IMU sensor includes both accelerometer and gyroscope
 - Accelerometer and gyroscope record the linear acceleration and angular velocity of the device respectively, which can be used for classification

Studying programming using Java/Javascript in Android Studio
 - Exploring APIs of the IMU sensor from VR headsets


=== Week 4: [[https://docs.google.com/presentation/d/1KR095rUYToCB6YsFtAxhQfv84Mugkq0Iz3CxqZKLWIs/edit?slide=id.g36300db7477_0_174#slide=id.g36300db7477_0_174|Slides]] 

More in depth analysis of papers
 - Motion sensor from VR: Real-Time Recognition of In-Place Body Actions and Head Gestures using Only a Head-Mounted Display [1]
 - Motion sensor from wearable devices: FitCoach: Virtual fitness coach empowered by wearable mobile devices [2]

In depth analysis in the inner workings of an IMU sensor

Accelerometer
 - Takes variance in capacitor charge to determine linear acceleration

 - Measures linear acceleration for 3 axes: x, y, and z 

Gyroscope
 - Small silicon mass vibrating at high Hertz.  

 - Coriolis effect applies, deflecting the silicon mass perpendicularly

 - Measures angular velocity around 3 axes: pitch, roll, and yaw


=== Week 5: [[|Slides]] 


----



----

Questions or contributions? Please contact us.

[1] Zhao, Jingbo, et al. "Real-time recognition of in-place body actions and head gestures using only a head-mounted display." 2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR). IEEE, 2023.

[2] Guo, Xiaonan, Jian Liu, and Yingying Chen. "FitCoach: Virtual fitness coach empowered by wearable mobile devices." IEEE INFOCOM 2017-IEEE Conference on Computer Communications. IEEE, 2017.