Resilient Edge-Cloud Autonomous Learning with Timely inferences

Project Advisor and Mentors: Professor Anand Sarwate, Professor Waheed Bajwa, Nitya Sathyavageeswaran, Vishakha Ramani, Yu Wu, Aliasghar Mohammadsalehi, Muhammad Zulqarnain

Team: Shreya Venugopaal, Haider Abdelrahman, Tanushree Mehta, Lakshya Gour, Yunhyuk Chang

Objective: The purpose of this project is to use ORBIT in order to design and run experiments which will analyze the training and prediction of various ML models across multiple edge devices. Additionally, students will develop a latency profiling framework for MEC-assisted machine learning using cutting edge techniques such as splitting models across multiple orbit nodes and networks, as well as early exit. They will then analyze these models for latency and accuracy tradeoff analysis, along with measuring network delays.

Week 1

Summary

• Understood the goal of the project and broke down its objectives
• Got familiar with Linux by practicing some simple Linux commands.
• Review some basic Python and other coding materials
• Understanding the basis of Machine Learning models and algorithms

Next Steps

• Create and train a “small” and “large” Neural Network
• Attempt to simulate the difference between their performances at inference

Week 2

Summary

• Performed basics of pattern recognition and Machine Learning (PPA - Patterns, Predictions, Actions) using Pytorch
• Created a node image with Pytorch
• Created small Machine Learning models
• Loaded the Modified National Institute of Standards and Technology (MNIST) database onto the orbit node

Next Steps

• Create and train a “small” and “large” Neural Network
• Attempt to simulate the difference between their performances at inference

Week 3

Summary

• Debugged issues with work we had done in previous weeks
• Connected 2 nodes with client-server architecture
• Extracted data for time and Accuracy measurements
• Added logging to code and stored logs in a table format (csv)
• Ready to start testing!

Next Steps

• Compute the delay time for each step to a higher precision
• Plot and identify trends in data (accuracy & latency)
• Begin to implement Split Computing - 10 layer MLP, ResNet
• Think about other implementations - Early Exiting, model compression, data compression, … Mixture?

Week 4

Summary

• Created and Trained a ‘Small’ and ’Large’ Neural Networks
• Compared their performances on the CIFAR10 dataset
• Established connection between two nodes
• Communicated test data between nodes to compare accuracy and delay between our NN models
  • Need to serialize by bytes instead of transferring as strings
• Read and discussed various research papers related to our project - created a brief presentation on each paper
  • James -
  • Shreya - <a href="​https://docs.google.com/presentation/d/1LvucmccciojbtaxkDGnkFUM0lYNFh7YuKeKjHB8zxjI/edit?usp=sharing">Mobile Edge Computing: A Survey</a>
  • Tanushree - <a href= "​https://docs.google.com/presentation/d/121ELfzNZW6dZzfScij3yEqSZ_zurKLX8nQJ05IO3Qeg/edit?usp=sharing">Distributed Deep Neural Networks over the Cloud, the Edge and End Devices</a>
  • Lakshya -
  • Haider -

Next Steps

• Divide data into “chunks” for faster and more efficient transmission
• Design experiments to test different architectures and implementations of Early Exiting and Split Computing
• Track and add Age of Information Metrics

Week 5

• ntp

Week 6

Summary

• Figured out how to properly split a NN using split computing
• First Experimented with image inferencing on the same device
• Split the Neural Network (ResNet 18) onto two different devices
• Ran an inference with the split NN across a network

Next Steps

• Take a step back
• Ask: what questions do we want to answer?
• As you vary the threshold for asking for help, how does the average latency change(over the dataset)?

Open ended

Transfer to Precision time protocol(PTP)

Links to Presentations

Week 1   Week 2   Week 3   Week 4   Week 5 - Cumulative   Week 6