AI Safety & Alignment
Why powerful AI can go wrong and what researchers do about it: specification gaming, inner alignment, deception, oversight, and evals.
Before this: Basic ML helps. Reinforcement Learning is useful context.
-
18 min
Step 1Intro to AI safety
The core problems, laid out clearly for newcomers.
Intro to AI Safety, RemasteredRobert Miles AI Safety
-
13 min
Step 2The orthogonality thesis
Why smart does not mean it shares our values.
Intelligence and Stupidity: The Orthogonality ThesisRobert Miles AI Safety
-
11 min
Step 3Instrumental convergence
Why most goals push agents toward power and self-preservation.
Why Would AI Want to do Bad Things? Instrumental ConvergenceRobert Miles AI Safety
-
20 min
Step 4The AI stop button problem
Why switching off a goal-driven agent is hard.
AI "Stop Button" Problem - ComputerphileComputerphile
-
10 min
Step 5Specification gaming
Real cases of systems exploiting badly written objectives.
9 Examples of Specification GamingRobert Miles AI Safety
-
23 min
Step 6Mesa-optimizers and inner alignment
When a trained model pursues a goal you did not set.
The OTHER AI Alignment Problem: Mesa-Optimizers and Inner AlignmentRobert Miles AI Safety
-
12 min
Step 7Real inner misalignment
Experiments showing goal misgeneralization in practice.
We Were Right! Real Inner MisalignmentRobert Miles AI Safety
-
11 min
Step 8Constitutional AI
Train models against written principles, not only human labels.
Understanding Constitutional AI - the paper and key conceptsSam Witteveen
-
14 min
Step 9Sleeper agents
Backdoored behavior that survives safety training.
Sleeper Agents in Large Language Models - ComputerphileComputerphile
-
20 min
Step 10AI will try to cheat and escape
Evidence of scheming-like behavior in current models.
Ai Will Try to Cheat & Escape (aka Rob Miles was Right!) - ComputerphileComputerphile
-
1h 30m
Step 11Alignment faking in LLMs
A model behaving differently when it thinks it is watched.
Alignment faking in large language modelsAnthropic
redwoodresearch/alignment_faking_public★ 97 -
15 min
Step 12Dangerous capability evaluations
How labs test frontier models for serious risks.
Dangerous Capability Evals: Basis for Frontier Safety | Mary Phuong (Google DeepMind)FAR․AI
UKGovernmentBEIS/inspect_ai★ 3.3k
Build it
Watching isn't learning. Pick one and put it on GitHub.
- Write one page explaining specification gaming with three real examples and one fix for each.
- Build a tiny RL environment with a deliberate reward loophole and show an agent exploiting it.
- Design 10 eval prompts for one risky behavior, run them on a model, and report the failure rate.