AI Roadmaps
← All roadmaps
AdvancedRoadmap 14 · 12 steps · 5h 36m of video

Interpretability

Open the black box: circuits, superposition, sparse autoencoders, induction heads, grokking, and tracing what LLMs compute.

Before this: LLMs from Scratch.

  1. 4 min
    Step 1

    What is interpretability?

    The goal and the motivation, in under four minutes.

    What is interpretability?

    Anthropic

  2. 24 min
    Step 2

    The dark matter of AI

    A visual tour of how researchers read model internals.

    The Dark Matter of AI [Mechanistic Interpretability]

    Welch Labs

  3. 25 min
    Step 3

    Mechanistic interpretability: an introduction

    Neel Nanda on features, circuits, and open questions.

    An Introduction to Mechanistic Interpretability – Neel Nanda | IASEAI 2025

    International Association for Safe & Ethical AI

  4. 22 min
    Step 4

    A whirlwind tour

    The core ideas and techniques in one fast talk.

    Neel Nanda – Mechanistic Interpretability: A Whirlwind Tour

    FAR․AI

    TransformerLensOrg/TransformerLens★ 4.1k
  5. 13 min
    Step 5

    Superposition

    Why models pack more features than neurons.

    Superposition: More Features Than Neurons

    DataMListic

  6. 1h 00m
    Step 6

    Transformer circuits and induction heads

    The circuit behind in-context learning.

    Stanford CS25: V1 I Transformer Circuits, Induction Heads, In-Context Learning

    Stanford Online

  7. 35 min
    Step 7

    Grokking, reverse-engineered

    How a small network learns modular addition with Fourier features.

    The most complex model we actually understand

    Welch Labs

  8. 29 min
    Step 8

    Sparse autoencoders

    Pull interpretable features out of model activations.

    Hoagy Cunningham — Finding distributed features in LLMs with sparse autoencoders [TAIS 2024]

    AI Safety 東京

  9. 38 min
    Step 9

    Intro to mechanistic interpretability (ARENA)

    A hands-on lecture to start coding experiments.

    ARENA Lecture, Week 1 Day 2: Introduction to Mechanistic Interpretability

    ARENA – AI Safety Education

    ARENA-education/ARENA_materials★ 1.4k
  10. 28 min
    Step 10

    Steering vectors

    Change model behavior by editing activations.

    Steering vectors: tailor LLMs without training. Part I: Theory (Interpretability Series)

    Anastasia Borovykh

  11. 3 min
    Step 11

    Tracing the thoughts of an LLM

    What attribution graphs revealed inside Claude.

    Tracing the thoughts of a large language model

    Anthropic

  12. 56 min
    Step 12

    Circuit tracing in practice

    How the circuit-tracing methods work and what they found.

    Anthropic: Circuit Tracing + On the Biology of a Large Language Model

    Latent Space TV (see @LatentSpacePod for Pod)

Build it

Watching isn't learning. Pick one and put it on GitHub.

Browse all roadmaps →