Run, Serve & Scale AI
Get models into production and make them fast: Docker, MLOps, monitoring, quantization, vLLM, GPUs, and FSDP.
Before this: Deep Learning. LLMs from Scratch for the inference steps.
-
2 min
-
26 min
Step 2Docker for an ML project
Package a model so it runs anywhere.
Docker Simply Explained with a Machine Learning Project for BeginnersPython Simplified
-
19 min
Step 3Serve a model with FastAPI
Turn a trained model into an HTTP API.
Deploy ML models with FastAPI, Docker, and Heroku | TutorialAssemblyAI
GokuMohandas/Made-With-ML★ 50.1k -
26 min
Step 4MLOps explained
Versioning, pipelines, deployment, and monitoring.
MLOps Explained - What It Is, Why You Need It and How It WorksTechWorld with Nana
DataTalksClub/mlops-zoomcamp★ 15.6k -
51 min
Step 5Track experiments with MLflow
Never lose the settings behind a good result.
MLFlow Tutorial | ML Ops Tutorialcodebasics
mlflow/mlflow★ 28.9k -
30 min
Step 6Monitor models for drift
Compare live inputs to training data and flag drift with Evidently reports.
Evidently AI Tutorial-Open Source ML Models Monitoring and ObservabilityKrish Naik
-
26 min
Step 7Quantization
8-bit, 4-bit, and GGUF: smaller models, small quality loss.
Quantizing LLMs - How & Why (8-Bit, 4-Bit, GGUF & More)Adam Lucek
-
9 min
Step 8llama.cpp
The engine that runs LLMs on laptops and phones.
What Is Llama.cpp? The LLM Inference Engine for Local AIIBM Technology
ggml-org/llama.cpp★ 135.8k -
9 min
Step 9The KV cache
Why inference memory grows with context length.
The KV Cache: Memory Usage in TransformersEfficient NLP
-
32 min
Step 10vLLM and PagedAttention
Serve many users at once without wasting GPU memory.
Fast LLM Serving with vLLM and PagedAttentionAnyscale
vllm-project/vllm★ 97.3k -
25 min
Step 11Speculative decoding
A small draft model proposes tokens; the big model verifies several in one pass.
How LLMs Get Faster Without Changing Their Outputs | Speculative DecodingJia-Bin Huang
-
10 min
Step 12Inference engines
How serving systems batch, cache, and schedule.
Inference Engines explained in 10min..Caleb Writes Code
-
28 min
Step 13How GPUs work
The hardware every model runs on, beautifully animated.
How do Graphics Cards Work? Exploring GPU ArchitectureBranch Education
-
1h 14m
Step 14GPUs for LLMs
Stanford CS336: memory hierarchy, arithmetic intensity, and why kernels matter.
Stanford CS336 I Language Modeling from Scratch | Spring 2025 | Lecture 5: GPUsStanford Online
gpu-mode/lectures★ 7.1k -
33 min
Step 15Sharded training with FSDP
Train models that don't fit on one GPU.
How Fully Sharded Data Parallel (FSDP) works?Ahmed Taha
stas00/ml-engineering★ 19.7k
Build it
Watching isn't learning. Pick one and put it on GitHub.
- Serve a quantized model with llama.cpp and compare tokens/sec at 4-bit and 8-bit.
- Load-test vLLM at 1, 8, and 32 concurrent users and chart time-to-first-token.
- Train a scikit-learn model, serve it with FastAPI, then shift the input data and catch the drift in an Evidently report.