AI Roadmaps
← All roadmaps
IntermediateRoadmap 27 · 12 steps · 4h 58m of video

Synthetic Data & Data-Centric AI

Improve models by improving their data: find label errors, label less, augment, generate synthetic tables and LLM training sets, and know when synthetic data backfires.

Before this: Machine Learning. Deep Learning helps for the GAN and VAE step; Fine-Tuning & Reasoning Models helps for the LLM steps.

  1. 26 min
    Step 1

    From model-centric to data-centric AI

    Andrew Ng's case for fixing data quality before tuning the model.

    Data centric AI development From Big Data to Good Data Andrew Ng

    Databricks

  2. 1h 13m
    Step 2

    Find label errors

    MIT lecture on confident learning: spot mislabeled examples automatically.

    Lecture 2: Label Errors

    Introduction to Data-Centric AI

    cleanlab/cleanlab★ 11.8k
  3. 1h 02m
    Step 3

    Dataset creation and curation

    MIT lecture on sourcing, labeling, and cleaning a dataset.

    Lecture 3: Dataset Creation and Curation

    Introduction to Data-Centric AI

  4. 6 min
    Step 4

    Active learning

    Let the model choose which examples are worth labeling next.

    Active (Machine) Learning - Computerphile

    Computerphile

  5. 30 min
    Step 5

    Weak supervision

    Combine noisy heuristic rules into training labels with Snorkel.

    Hand in hand with weak supervision using snorkel - Szymon Wojciechowski

    PyData

  6. 10 min
    Step 6

    Data augmentation

    Flips, crops, and color shifts that stretch a small image dataset.

    C4W2L10 Data Augmentation

    DeepLearningAI

  7. 7 min
    Step 7

    What synthetic data is

    Generated data that keeps real data's patterns, and where it helps.

    What is Synthetic Data? No, It's Not "Fake" Data

    IBM Technology

  8. 9 min
    Step 8

    Synthetic tables with SDV

    Generate single- and multi-table data with copulas and GANs.

    Synthetic Data Generation with SDV

    CeADAR Ireland

    sdv-dev/SDV★ 3.6k
  9. 35 min
    Step 9

    CTGAN and TVAE in Python

    The theory behind two tabular generators, then code using both.

    Synthetic Data Generation with Generative Models in Python

    NeuralNine

  10. 3 min
    Step 10

    Synthetic datasets for LLM fine-tuning

    Turn your documents into QA and reasoning pairs with a CLI.

    How to Create Synthetic Datasets for Fine-Tuning Llama

    Meta Developers

    meta-llama/synthetic-data-kit★ 1.7k
  11. 19 min
    Step 11

    Curating data for LLM post-training

    Lessons from building the OpenThoughts reasoning dataset.

    Data and Environment Curation for Post-Training LLMs — Mahesh Sathiamoorthy, Bespoke Labs

    AI Engineer

  12. 18 min
    Step 12

    Model collapse

    What can go wrong when models train on model-generated data.

    What Happens When All Training Data is AI Generated?

    Mutual Information

Build it

Watching isn't learning. Pick one and put it on GitHub.

Browse all roadmaps →