Text Books
Course Content
NOTE: Topics below are tentative (until we are past that week).
We will update it as we go through the lectures in each week.
| Topic |
Slides |
|
| Introduction |
intro.pdf |
|
| Linux scheduling |
scheduling1.pdf, scheduling2.pdf |
|
| Real Time Systems |
realtime1.pdf, realtime2.pdf, realtime3.pdf |
|
| Hardware platforms, datasheets, IDE (STM32Cube), device drivers, hal/ll |
Cortex M4 reference manual,
Cortex M4 user manual,
STM32 Cube IDE,
Linux Device Driver,
STM32 HAL LL libraries
|
|
| Machine Learning Model Compression for Inferences on Embedded Platforms |
(i) Quantization stm32cubeai.pdf, cnn-basics.pdf, quantization.pdf,
(ii) Pruning slides, pruning_architecture_paper, pruning_energy_paper, pruning_gpu_issues |
|
| Small Machine Learning Model Architectures for Embedded Platforms |
(i) 1x1 bottleneck layer inception-1x1-bottlenecklayer.pdf, Andrew NG video
(ii) depthwise separable convolutions mobilenet-depthwise-separable-convolution.pdf
(iii) Compact tree architecture with non-linearities like NN Bonsai_paper.pdf
|
|
| Introduction to security through some demonstrations with Metasploit framework on Kali Linux |
(i) System security (virus example for reverse_tcp connection on Windows) and network security (nMap for port scanning, ssh and tomcat brute force attacks on Metasploitable) system_network_security.pdf
(ii) Mobile security mobile_security.pdf
|
|
Project Part 1
Project Part 1 Goal: Detect car horns on Sensortile M4 micro-controller at high accuracy and low latency (not missing audio)
Tutorials and resources:
Audio event detection on Sensortile Cortex M4
(a) Train a model including the class car_horn
model training
(b) Evaluate quantized model accuracy on laptop
model quantization and evaluation
(c) Evaluate deployed model latency on Sensortile M4
model deployment
This first part of the project will be marked on (i) working project demo, (ii) git repo with readme for training and deployment on Sensortile M4, (iii) extra credit: train more than one model and compare accuracy vs. latency. Deadline is Oct 8.
Project Part 2
Project Part 2 Goal: Learn how Machine Learning Core (MLC) works on STM sensors
Tutorials and resources:
MLC core on STM Sensors. The SensorTile.box has the LSM6DSOX inertial sensor module. This can sample sensors like 3-axis accelerometer and 3-axis gyroscope, process the sensor data with filters, extract features from time windows of data and use the features to run decision tree classifiers and detect events. All these are done in hardware, without involvement of the Cortex M4 CPU core. Interrupts are sent to the CPU core, based on the events detected.
Applications shows a list of applications that follow this pipeline on LSM6DSOX module.
Configurations shows a set of software tools like the ST Ble Sensor Android app, STM32Cube Programmer, Unicleo, Unico, Algobuilder, and their configurations and flows with different hardware platforms to train/test MLC. Some slide decks descrbing the tools and workflow are
workflow1, and
workflow2.
Todo:
Reproduce
Yoga pose estimation with Sensortilebox, STBLE app and Unico, as Unico works on Linux laptops. You can use Unicleo/AlgoBuilder if you have windows machine. You can collect and label you own training data for fun, or use the datalogs given. But you must load these logs in Unico, train the decision tree and deploy the UCF back on SensorTile.box to run.
Deadline is Nov 17 and demos for both parts will be taken after major exams. If you are done early, try to run both parts on the newer board together, and measure latencies.