Track

Deep Learning

Goal: Build and train neural networks from perceptrons through convolutional and recurrent architectures.

Prereqs: ML Basics and Math for ML.

Status: done

Work through the steps in order. Bold links open YouTube.

Step	Concept	YouTube	Read
1	Perceptron and the neuron model	3Blue1Brown — But what is a neural network?	DL Book — Introduction (ch. 1)
2	Gradient descent	3Blue1Brown — Gradient descent, how neural networks learn	DL Book — Numerical computation (ch. 4)
3	Backpropagation	Karpathy — The spelled-out intro to neural networks and backpropagation: building micrograd	cs231n — Backprop in practice
4	Deep feedforward networks	3Blue1Brown — What is backpropagation really doing?	DL Book — Deep feedforward networks (ch. 6)
5	Activations, gradients, batch norm	Karpathy — Building makemore Part 3: Activations & Gradients, BatchNorm	Ioffe & Szegedy 2015 — Batch Normalization
6	Regularisation: dropout	Karpathy — Building makemore Part 4: Becoming a Backprop Ninja	DL Book — Regularization for deep learning (ch. 7)
7	Adam optimiser	StatQuest — Adam Optimizer	Kingma & Ba 2014 — Adam
8	Convolutional networks	3Blue1Brown — But what is a convolution?	cs231n — Convolutional neural networks
9	ResNets and skip connections	Karpathy — Let’s reproduce GPT-2 (ResNet segment)	He et al. 2015 — Deep Residual Learning
10	Recurrent networks and BPTT	Andrej Karpathy — The unreasonable effectiveness of RNNs	cs231n — Recurrent neural networks
11	Layer normalisation		Ba et al. 2016 — Layer Normalization
12	Training loop in PyTorch	Patrick Loeber — PyTorch Tutorial — Training pipeline	PyTorch — Training a classifier

Found a broken link or an unclear step? Report a problem with this track.