Fine-Tuning & Reasoning Models
Adapt open models with LoRA, align them with RLHF and DPO, and train reasoning with GRPO.
Before this: LLMs from Scratch.
-
11 min
Step 1Do you even need fine-tuning?
Prompting, RAG, or LoRA: choose before you spend GPU hours.
Is Fine-Tuning Still Needed? LLMs, RAG, & LoRAIBM Technology
-
7 min
Step 2LoRA, by its inventor
Why training a tiny low-rank update is enough.
What is Low-Rank Adaptation (LoRA) | explained by the inventorEdward Hu
huggingface/peft★ 21.9k -
39 min
Step 3Fine-tune step by step
Data, tokenization, loss masking, and training with Hugging Face.
Finetune LLMs to teach them ANYTHING with Huggingface and Pytorch | Step-by-step tutorialNeural Breakdown with AVB
huggingface/trl★ 19.8k -
33 min
Step 4Fine-tune on a free GPU
Unsloth makes it fast and memory-light, then run it in Ollama.
Fine-Tuning Local LLMs with Unsloth & OllamaNeuralNine
unslothai/unsloth★ 79.8k -
18 min
Step 5RLHF
How human preferences turned base models into assistants.
Reinforcement Learning with Human Feedback (RLHF), Clearly Explained!!!StatQuest with Josh Starmer
-
9 min
Step 6DPO
Preference tuning without a reward model or RL loop.
Direct Preference Optimization: Your Language Model is Secretly a Reward Model | DPO paper explainedAI Coffee Break with Letitia
-
13 min
Step 7Why reasoning models think
Test-time compute: spending more effort on harder questions.
How do thinking and reasoning models work?Google for Developers
-
23 min
Step 8GRPO, visually
The RL algorithm behind DeepSeek's reasoning models.
DeepSeek's GRPO (Group Relative Policy Optimization) | Reinforcement Learning for LLMsJulia Turc
-
1h 09m
Step 9GRPO, the paper
A deep read of DeepSeekMath with the math explained.
[GRPO Explained] DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language ModelsYannic Kilcher
-
9 min
Step 10DeepSeek-R1
How pure RL with checkable rewards produced a reasoning model.
DeepSeek-R1 Paper Explained - A New RL LLMs Era in AI?AI Papers Academy
huggingface/open-r1★ 26.5k -
2h 42m
Step 11RL for LLMs, hands on
A workshop covering RL, reasoning, kernels, and quantization.
[Full Workshop] Reinforcement Learning, Kernels, Reasoning, Quantization & Agents — Daniel HanAI Engineer
mlabonne/llm-course★ 85.2k
Build it
Watching isn't learning. Pick one and put it on GitHub.
- Fine-tune a 0.5B–3B model with QLoRA on 1,000 examples of a narrow task and compare it with the base model on 50 held-out cases.
- Train a small model with GRPO on GSM8K using exact-match rewards; plot reward, response length, and accuracy.
- Collect 300 chosen/rejected pairs and run DPO on top of your fine-tuned model.