Deep Learning
Neural networks from the inside: backprop, build your own autograd, CNNs, RNNs, and PyTorch.
Before this: Python & Math for AI. Machine Learning helps.
-
19 min
Step 1What is a neural network?
The clearest 20 minutes on neurons, weights, and layers.
But what is a neural network? | Deep learning chapter 13Blue1Brown
-
21 min
Step 2How networks learn
Gradient descent applied to millions of weights.
Gradient descent, how neural networks learn | Deep Learning Chapter 23Blue1Brown
-
13 min
Step 3Backpropagation, intuitively
What the algorithm is really doing.
Backpropagation, intuitively | Deep Learning Chapter 33Blue1Brown
-
10 min
Step 4Backpropagation calculus
The chain rule that makes it work.
Backpropagation calculus | Deep Learning Chapter 43Blue1Brown
-
2h 26m
Step 5Build an autograd engine
Write backprop yourself in about 100 lines. Nothing beats this.
The spelled-out intro to neural networks and backpropagation: building microgradAndrej Karpathy
karpathy/micrograd★ 19k -
1h 58m
Step 6Your first language model
A bigram model, then a neural one, from a blank file.
The spelled-out intro to language modeling: building makemoreAndrej Karpathy
karpathy/nn-zero-to-hero★ 25.9k -
1h 16m
Step 7Multilayer perceptrons
Embeddings, hidden layers, and training splits done properly.
Building makemore Part 2: MLPAndrej Karpathy
-
1h 56m
Step 8Activations, gradients, BatchNorm
Why deep nets fail to train and how to fix it.
Building makemore Part 3: Activations & Gradients, BatchNormAndrej Karpathy
-
23 min
Step 9Optimizers: SGD to Adam
What momentum and Adam actually change.
Who's Adam and What's He Optimizing? | Deep Dive into Optimizers for Machine Learning!Sourish Kundu
-
23 min
Step 10Convolutions
The operation behind image models, explained visually.
But what is a convolution?3Blue1Brown
-
17 min
Step 11Recurrent neural networks
How networks handle sequences, and why they struggle.
Recurrent Neural Networks (RNNs), Clearly Explained!!!StatQuest with Josh Starmer
-
21 min
Step 12LSTMs
The gating trick that made RNNs useful.
Long Short-Term Memory (LSTM), Clearly ExplainedStatQuest with Josh Starmer
-
10 min
Step 13Residual connections
The skip connection that lets networks go hundreds of layers deep.
ResNet (actually) explained in under 10 minutesrupert ai
-
50 min
Step 14PyTorch crash course
Tensors, autograd, models, and the training loop in under an hour. The repo has a full free course when you want more.
PyTorch Crash Course - Getting Started with Deep LearningAssemblyAI
mrdbourke/pytorch-deep-learning★ 19.4k
Build it
Watching isn't learning. Pick one and put it on GitHub.
- Extend micrograd with ReLU and a softmax loss, then train it on a toy dataset.
- Train an MLP on MNIST in raw PyTorch, then add BatchNorm and dropout one at a time and record what changes.
- Train a small ResNet on CIFAR-10 with and without skip connections and compare the curves.