AI Roadmaps
← All roadmaps
IntermediateRoadmap 32 · 12 steps · 5h 27m of video

LLM Security & Red Teaming

Learn how LLM apps get attacked, from prompt injection and jailbreaks to data extraction and poisoning, then test and defend your own.

Before this: AI Engineering. AI Agents & MCP helps for the agent and tool-use steps.

  1. 25 min
    Step 1

    The OWASP Top 10 for LLMs

    A map of the main ways LLM applications get attacked.

    OWASP's Top 10 Ways to Attack LLMs: AI Vulnerabilities Exposed

    IBM Technology

  2. 12 min
    Step 2

    Prompt injection

    Simon Willison on why mixing instructions and untrusted text is unsafe.

    Prompt Injection, explained

    Simon Willison

  3. 12 min
    Step 3

    Jailbreaks

    Computerphile on how prompts talk models past their safety training.

    ChatGPT Jailbreak - Computerphile

    Computerphile

  4. 31 min
    Step 4

    Automated adversarial suffixes

    Zico Kolter on optimized suffixes that jailbreak many aligned models.

    Zico Kolter - Adversarial Attacks on Aligned Language Models

    FAR․AI

  5. 49 min
    Step 5

    Indirect prompt injection in the wild

    Real exploits where web pages and documents hijack LLM apps.

    Indirect Prompt Injections in the Wild – Real World exploits and mitigations Johann Rehberger

    Ekoparty Security Conference

  6. 11 min
    Step 6

    Extracting training data

    USENIX talk showing GPT-2 can regurgitate memorized training text.

    USENIX Security '21 - Extracting Training Data from Large Language Models

    USENIX

  7. 12 min
    Step 7

    Backdoors and sleeper agents

    Anthropic's study of hidden trigger behavior that survived safety training.

    AI Sleeper Agents: How Anthropic Trains and Catches Them

    Rational Animations

  8. 34 min
    Step 8

    Design patterns for secure agents

    Limit what an agent can do after it reads untrusted input.

    LLM Chronicles #6.9: Design Patterns for Securing LLM Agents Against Prompt Injection (Paper Review)

    Donato Capitella

  9. 21 min
    Step 9

    Guardrails

    Add input and output rails to a chatbot with NeMo Guardrails.

    NVIDIA NeMo Guardrails: Full Walkthrough for Chatbots / AI

    James Briggs

  10. 25 min
    Step 10

    Automated red teaming with PyRIT

    Microsoft's open tool for sending attack prompts and scoring replies.

    BlueHat 2024: S24: Automate AI Red Teaming in your existing tool chain with PyRIT

    Microsoft Security Response Center (MSRC)

    microsoft/PyRIT★ 5k
  11. 19 min
    Step 11

    Scan a model with garak

    Run garak's probes against your own chatbot and read the report.

    LLM Vulnerability Scanning with garak. Tutorial: Test your own chat bots!

    Embrace The Red

    NVIDIA/garak★ 10.1k
  12. 1h 15m
    Step 12

    Defending against jailbreaks

    Anthropic researchers on classifier defenses and their red-teaming results.

    Defending against AI jailbreaks

    Anthropic

Build it

Watching isn't learning. Pick one and put it on GitHub.

Browse all roadmaps →