AI Roadmaps
← All roadmaps
IntermediateRoadmap 15 · 12 steps · 4h 17m of video

AI Safety & Alignment

Why powerful AI can go wrong and what researchers do about it: specification gaming, inner alignment, deception, oversight, and evals.

Before this: Basic ML helps. Reinforcement Learning is useful context.

  1. 18 min
    Step 1

    Intro to AI safety

    The core problems, laid out clearly for newcomers.

    Intro to AI Safety, Remastered

    Robert Miles AI Safety

  2. 13 min
    Step 2

    The orthogonality thesis

    Why smart does not mean it shares our values.

    Intelligence and Stupidity: The Orthogonality Thesis

    Robert Miles AI Safety

  3. 11 min
    Step 3

    Instrumental convergence

    Why most goals push agents toward power and self-preservation.

    Why Would AI Want to do Bad Things? Instrumental Convergence

    Robert Miles AI Safety

  4. 20 min
    Step 4

    The AI stop button problem

    Why switching off a goal-driven agent is hard.

    AI "Stop Button" Problem - Computerphile

    Computerphile

  5. 10 min
    Step 5

    Specification gaming

    Real cases of systems exploiting badly written objectives.

    9 Examples of Specification Gaming

    Robert Miles AI Safety

  6. 23 min
    Step 6

    Mesa-optimizers and inner alignment

    When a trained model pursues a goal you did not set.

    The OTHER AI Alignment Problem: Mesa-Optimizers and Inner Alignment

    Robert Miles AI Safety

  7. 12 min
    Step 7

    Real inner misalignment

    Experiments showing goal misgeneralization in practice.

    We Were Right! Real Inner Misalignment

    Robert Miles AI Safety

  8. 11 min
    Step 8

    Constitutional AI

    Train models against written principles, not only human labels.

    Understanding Constitutional AI - the paper and key concepts

    Sam Witteveen

  9. 14 min
    Step 9

    Sleeper agents

    Backdoored behavior that survives safety training.

    Sleeper Agents in Large Language Models - Computerphile

    Computerphile

  10. 20 min
    Step 10

    AI will try to cheat and escape

    Evidence of scheming-like behavior in current models.

    Ai Will Try to Cheat & Escape (aka Rob Miles was Right!) - Computerphile

    Computerphile

  11. 1h 30m
    Step 11

    Alignment faking in LLMs

    A model behaving differently when it thinks it is watched.

    Alignment faking in large language models

    Anthropic

    redwoodresearch/alignment_faking_public★ 97
  12. 15 min
    Step 12

    Dangerous capability evaluations

    How labs test frontier models for serious risks.

    Dangerous Capability Evals: Basis for Frontier Safety | Mary Phuong (Google DeepMind)

    FAR․AI

    UKGovernmentBEIS/inspect_ai★ 3.3k

Build it

Watching isn't learning. Pick one and put it on GitHub.

Browse all roadmaps →