AI Roadmaps
← All roadmaps
AdvancedRoadmap 07 · 11 steps · 6h 35m of video

Fine-Tuning & Reasoning Models

Adapt open models with LoRA, align them with RLHF and DPO, and train reasoning with GRPO.

Before this: LLMs from Scratch.

  1. 11 min
    Step 1

    Do you even need fine-tuning?

    Prompting, RAG, or LoRA: choose before you spend GPU hours.

    Is Fine-Tuning Still Needed? LLMs, RAG, & LoRA

    IBM Technology

  2. 7 min
    Step 2

    LoRA, by its inventor

    Why training a tiny low-rank update is enough.

    What is Low-Rank Adaptation (LoRA) | explained by the inventor

    Edward Hu

    huggingface/peft★ 21.9k
  3. 39 min
    Step 3

    Fine-tune step by step

    Data, tokenization, loss masking, and training with Hugging Face.

    Finetune LLMs to teach them ANYTHING with Huggingface and Pytorch | Step-by-step tutorial

    Neural Breakdown with AVB

    huggingface/trl★ 19.8k
  4. 33 min
    Step 4

    Fine-tune on a free GPU

    Unsloth makes it fast and memory-light, then run it in Ollama.

    Fine-Tuning Local LLMs with Unsloth & Ollama

    NeuralNine

    unslothai/unsloth★ 79.8k
  5. 18 min
    Step 5

    RLHF

    How human preferences turned base models into assistants.

    Reinforcement Learning with Human Feedback (RLHF), Clearly Explained!!!

    StatQuest with Josh Starmer

  6. 9 min
    Step 6

    DPO

    Preference tuning without a reward model or RL loop.

    Direct Preference Optimization: Your Language Model is Secretly a Reward Model | DPO paper explained

    AI Coffee Break with Letitia

  7. 13 min
    Step 7

    Why reasoning models think

    Test-time compute: spending more effort on harder questions.

    How do thinking and reasoning models work?

    Google for Developers

  8. 23 min
    Step 8

    GRPO, visually

    The RL algorithm behind DeepSeek's reasoning models.

    DeepSeek's GRPO (Group Relative Policy Optimization) | Reinforcement Learning for LLMs

    Julia Turc

  9. 1h 09m
    Step 9

    GRPO, the paper

    A deep read of DeepSeekMath with the math explained.

    [GRPO Explained] DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

    Yannic Kilcher

  10. 9 min
    Step 10

    DeepSeek-R1

    How pure RL with checkable rewards produced a reasoning model.

    DeepSeek-R1 Paper Explained - A New RL LLMs Era in AI?

    AI Papers Academy

    huggingface/open-r1★ 26.5k
  11. 2h 42m
    Step 11

    RL for LLMs, hands on

    A workshop covering RL, reasoning, kernels, and quantization.

    [Full Workshop] Reinforcement Learning, Kernels, Reasoning, Quantization & Agents — Daniel Han

    AI Engineer

    mlabonne/llm-course★ 85.2k

Build it

Watching isn't learning. Pick one and put it on GitHub.

Where to go next