Reinforcement Learning
Agents that learn by trial and error: from Bellman equations and Q-learning to PPO, AlphaGo, and the RL behind reasoning models.
Before this: Deep Learning. Fine-Tuning & Reasoning Models pairs well at the end.
-
18 min
Step 1Reinforcement learning, the essentials
Rewards, policies, and the learning loop in plain language.
Reinforcement Learning: Essential ConceptsStatQuest with Josh Starmer
-
18 min
Step 2The RL framework, by the book
States, actions, returns, and how the pieces fit together.
Reinforcement Learning, by the BookMutual Information
openai/spinningup★ 12k -
22 min
Step 3Bellman equations and dynamic programming
The recursion every value-based method builds on.
Bellman Equations, Dynamic Programming, Generalized Policy Iteration | Reinforcement Learning Part 2Mutual Information
-
12 min
Step 4Q-learning
Learn action values from experience without a model.
Q-learning - Explained!CodeEmporium
-
11 min
Step 5Deep Q-learning
Replace the Q-table with a neural network.
Deep Q-Learning - Combining Neural Networks and Reinforcement Learningdeeplizard
huggingface/deep-rl-class★ 5.1k -
29 min
Step 6Policy gradients
Optimize the policy directly instead of values.
Policy Gradient Methods | Reinforcement Learning Part 6Mutual Information
-
31 min
Step 7PPO, step by step
The stable default policy-gradient algorithm, explained simply.
Simply Explaining Proximal Policy Optimization (PPO) | Deep Reinforcement LearningJohnny Code
DLR-RM/stable-baselines3★ 14k -
13 min
Step 8Get started with Stable Baselines3
Train your first agent on a Gymnasium environment in Python.
Getting Started with Stable Baselines3 - Lowest Learning Curve Python Reinforcement Learning LibraryJohnny Code
Farama-Foundation/Gymnasium★ 12.8k -
8 min
Step 9AlphaGo
How RL plus tree search beat a Go champion.
Reinforcement Learning: AlphaGoGraphics in 5 Minutes
-
22 min
Step 10PPO for language models
How PPO adapts to training LLMs with feedback.
Proximal Policy Optimization (PPO) for LLMs Explained IntuitivelyJulia Turc
huggingface/trl★ 19.8k -
24 min
Step 11GRPO and reasoning RL
The simpler RL recipe behind DeepSeek-R1-style reasoning.
GRPO: The RL Trick Behind DeepSeek-R1's ReasoningJia-Bin Huang
Build it
Watching isn't learning. Pick one and put it on GitHub.
- Train a DQN on CartPole, then plot reward per episode and compare against a random agent.
- Solve LunarLander with PPO in Stable Baselines3 and record a video of the trained agent.
- Write tabular Q-learning for FrozenLake from scratch and visualize the learned policy.