TinyML & Edge AI
Shrink models and run them on microcontrollers, phones, and small boards: pruning, quantization, distillation, TFLite Micro, Edge Impulse, ONNX, and ExecuTorch.
Before this: Deep Learning (training a CNN in PyTorch or Keras). Hardware steps use a microcontroller board such as an ESP32-CAM or Arduino Nano 33 BLE Sense, a Jetson, or a Raspberry Pi; the rest run on a laptop.
-
6 min
-
1h 03m
Step 2Why efficient ML
MIT's course intro: the gap between model size and hardware.
EfficientML.ai Lecture 1 - Introduction (MIT 6.5940 Fall 2026)MIT HAN Lab
-
42 min
Step 3Pruning and sparsity
Remove weights a model does not need, and keep accuracy.
EfficientML.ai Lecture 3 - Pruning and Sparsity (Part I) (MIT 6.5940 Fall 2026)MIT HAN Lab
-
23 min
Step 4int8 quantization
Store weights and activations in 8 bits instead of 32.
Understanding int8 neural network quantizationOscar Savolainen
-
6 min
Step 5Knowledge distillation
Train a small student model to copy a big teacher.
Knowledge Distillation - ExplainedDataMListic
-
1h 03m
Step 6TinyML on microcontrollers
MCUNet fits vision models into kilobytes of memory.
EfficientML.ai Lecture 10 - MCUNet and TinyML (MIT 6.5940 Fall 2026)MIT HAN Lab
mit-han-lab/tinyengine★ 969 -
13 min
Step 7Optimize TensorFlow Lite models
Post-training quantization and pruning in the TFLite toolchain.
Optimize your TensorFlow Lite models | SessionTensorFlow
-
19 min
Step 8TensorFlow Lite for Microcontrollers
Deploy a first model to a microcontroller, end to end.
TinyML: Getting Started with TensorFlow Lite for Microcontrollers | Digi-Key ElectronicsDigiKey
tensorflow/tflite-micro★ 3.2k -
28 min
Step 9Keyword spotting in 30 KB of RAM
Collect audio, train, and deploy a wake-word model.
Build Your Own ML-Powered Keyword Spotting Model in 30KB RAMEdge Impulse
-
17 min
Step 10Object detection on an ESP32-CAM
Run a tiny detector on a cheap camera board.
How to do Object Detection using ESP32-CAM and Edge Impulse YOLO ModelCircuit Digest
-
8 min
Step 11ONNX Runtime for generative models
See how ONNX Runtime GenAI runs language models on a device.
Introduction to ONNX RuntimeNVIDIA Developer
-
25 min
Step 12On-device PyTorch with ExecuTorch
PyTorch's runtime for phones and embedded chips.
ExecuTorch 1.0: General Availability Status for Mobile and Embedded...- Mergen Nachin & Cemal BilginPyTorch Foundation
pytorch/executorch★ 5.2k -
5 min
Step 13Get started with a Jetson Orin Nano
Set up a small GPU board for edge vision and LLMs.
Getting Started with the Jetson Orin Nano Developer KitNVIDIA Developer
-
9 min
Step 14Run an LLM on a Raspberry Pi 5
What a small language model can do on a cheap single-board computer.
Gemma 4 on Raspberry Pi 5: A Surprisingly Usable Local AI SetupZero to MVP
ggml-org/llama.cpp★ 135.8k
Build it
Watching isn't learning. Pick one and put it on GitHub.
- Quantize a small CNN to int8 and report size, latency, and accuracy before and after.
- Train a two-word keyword spotter and run it on a microcontroller board (or the board's simulator).
- Run a small language model locally with ONNX Runtime GenAI and measure its tokens per second on your laptop.