AI Roadmaps
← All roadmaps
AdvancedRoadmap 23 · 15 steps · 7h 04m of video

Bayesian ML & Probabilistic Programming

Reason with uncertainty: priors and posteriors, MCMC, PyMC, hierarchical models, Gaussian processes, variational inference, and Bayesian deep learning.

Before this: Machine Learning and Math for ML, Deeper (probability distributions, MLE, the multivariate Gaussian, KL divergence).

  1. 13 min
    Step 1

    Probabilities of probabilities

    Treat an unknown rate as something you can be uncertain about.

    Binomial distributions | Probabilities of probabilities, part 1

    3Blue1Brown

  2. 1h 37m
    Step 2

    Bayesian updating by counting

    McElreath builds a posterior from counted ways the data could occur.

    Statistical Rethinking 2023 - 02 - The Garden of Forking Data

    Richard McElreath

  3. 18 min
    Step 3

    Conjugate priors

    Update a Beta prior with data in closed form.

    Bayesian Updates and Conjugate Priors

    Steve Brunton

  4. 18 min
    Step 4

    MLE vs MAP

    How adding a prior turns maximum likelihood into MAP.

    What are Maximum Likelihood (ML) and Maximum a posteriori (MAP)? ("Best explanation on YouTube")

    Iain Explains Signals, Systems, and Digital Comms

  5. 16 min
    Step 5

    Bayesian linear regression

    Regression with a distribution over weights, not one answer.

    Bayesian Linear Regression : Data Science Concepts

    ritvikmath

  6. 12 min
    Step 6

    Markov chain Monte Carlo

    Sample a posterior known only up to its normalizing constant.

    Markov Chain Monte Carlo (MCMC) : Data Science Concepts

    ritvikmath

  7. 18 min
    Step 7

    Metropolis-Hastings

    The classic accept-or-reject MCMC sampler.

    Metropolis - Hastings : Data Science Concepts

    ritvikmath

  8. 32 min
    Step 8

    Hamiltonian Monte Carlo

    Use gradients to make big, efficient sampler moves.

    The intuition behind the Hamiltonian Monte Carlo algorithm

    Ben Lambert

  9. 1h 27m
    Step 9

    Bayesian modeling with PyMC

    Write a model in Python and let PyMC sample it.

    Chris Fonnesbeck - Probabilistic Python: An Introduction to Bayesian Modeling with PyMC

    PyData

    pymc-devs/pymc★ 9.9k
  10. 11 min
    Step 10

    Posterior predictive checks

    Simulate fake data from the model to see if it fits.

    What is a posterior predictive check and why is it useful?

    Ben Lambert

    arviz-devs/arviz★ 1.9k
  11. 13 min
    Step 11

    Hierarchical models

    Share strength across groups with partial pooling.

    Statistical Gold Nuggets | Bayesian Hierarchical Models

    Very Normal

  12. 24 min
    Step 12

    Gaussian processes

    A prior over whole functions, with error bars for free.

    Gaussian Processes

    Mutual Information

    cornellius-gp/gpytorch★ 3.9k
  13. 8 min
    Step 13

    Bayesian optimization

    Tune expensive black boxes with a GP and an acquisition function.

    Bayesian Optimization

    DataMListic

    bayesian-optimization/BayesianOptimization★ 8.7k
  14. 25 min
    Step 14

    Variational inference and the ELBO

    Turn inference into optimization when sampling is too slow.

    Variational Inference | Evidence Lower Bound (ELBO) | Intuition & Visualization

    Machine Learning & Simulation

    pyro-ppl/numpyro★ 2.8k
  15. 31 min
    Step 15

    Bayesian neural networks

    Get uncertainty estimates out of deep networks.

    Uncertainty estimation and Bayesian Neural Networks - Marcin Możejko

    PyData

Build it

Watching isn't learning. Pick one and put it on GitHub.

Browse all roadmaps →