AI Roadmaps
← All roadmaps
IntermediateRoadmap 03 · 14 steps · 11h 01m of video

Deep Learning

Neural networks from the inside: backprop, build your own autograd, CNNs, RNNs, and PyTorch.

Before this: Python & Math for AI. Machine Learning helps.

  1. 19 min
    Step 1

    What is a neural network?

    The clearest 20 minutes on neurons, weights, and layers.

    But what is a neural network? | Deep learning chapter 1

    3Blue1Brown

  2. 21 min
    Step 2

    How networks learn

    Gradient descent applied to millions of weights.

    Gradient descent, how neural networks learn | Deep Learning Chapter 2

    3Blue1Brown

  3. 13 min
    Step 3

    Backpropagation, intuitively

    What the algorithm is really doing.

    Backpropagation, intuitively | Deep Learning Chapter 3

    3Blue1Brown

  4. 10 min
    Step 4

    Backpropagation calculus

    The chain rule that makes it work.

    Backpropagation calculus | Deep Learning Chapter 4

    3Blue1Brown

  5. 2h 26m
    Step 5

    Build an autograd engine

    Write backprop yourself in about 100 lines. Nothing beats this.

    The spelled-out intro to neural networks and backpropagation: building micrograd

    Andrej Karpathy

    karpathy/micrograd★ 19k
  6. 1h 58m
    Step 6

    Your first language model

    A bigram model, then a neural one, from a blank file.

    The spelled-out intro to language modeling: building makemore

    Andrej Karpathy

    karpathy/nn-zero-to-hero★ 25.9k
  7. 1h 16m
    Step 7

    Multilayer perceptrons

    Embeddings, hidden layers, and training splits done properly.

    Building makemore Part 2: MLP

    Andrej Karpathy

  8. 1h 56m
    Step 8

    Activations, gradients, BatchNorm

    Why deep nets fail to train and how to fix it.

    Building makemore Part 3: Activations & Gradients, BatchNorm

    Andrej Karpathy

  9. 23 min
    Step 9

    Optimizers: SGD to Adam

    What momentum and Adam actually change.

    Who's Adam and What's He Optimizing? | Deep Dive into Optimizers for Machine Learning!

    Sourish Kundu

  10. 23 min
    Step 10

    Convolutions

    The operation behind image models, explained visually.

    But what is a convolution?

    3Blue1Brown

  11. 17 min
    Step 11

    Recurrent neural networks

    How networks handle sequences, and why they struggle.

    Recurrent Neural Networks (RNNs), Clearly Explained!!!

    StatQuest with Josh Starmer

  12. 21 min
    Step 12

    LSTMs

    The gating trick that made RNNs useful.

    Long Short-Term Memory (LSTM), Clearly Explained

    StatQuest with Josh Starmer

  13. 10 min
    Step 13

    Residual connections

    The skip connection that lets networks go hundreds of layers deep.

    ResNet (actually) explained in under 10 minutes

    rupert ai

  14. 50 min
    Step 14

    PyTorch crash course

    Tensors, autograd, models, and the training loop in under an hour. The repo has a full free course when you want more.

    PyTorch Crash Course - Getting Started with Deep Learning

    AssemblyAI

    mrdbourke/pytorch-deep-learning★ 19.4k

Build it

Watching isn't learning. Pick one and put it on GitHub.

Where to go next