AI Roadmaps
← All roadmaps
IntermediateRoadmap 22 · 14 steps · 3h 06m of video

Math for ML, Deeper

The next layer of math behind ML papers: change of basis, SVD, Jacobians, Hessians, convexity, Lagrange multipliers, Gaussians, and KL divergence.

Before this: Python & Math for AI (vectors, matrices, derivatives, Bayes' theorem, the normal distribution).

  1. 10 min
    Step 1

    Span and basis vectors

    What it means for vectors to build a whole space.

    Linear combinations, span, and basis vectors | Chapter 2, Essence of linear algebra

    3Blue1Brown

  2. 14 min
    Step 2

    Dot products and duality

    Why a dot product is secretly a projection and a linear map.

    Dot products and duality | Chapter 9, Essence of linear algebra

    3Blue1Brown

  3. 13 min
    Step 3

    Change of basis

    Describe the same vector in different coordinate systems.

    Change of basis | Chapter 13, Essence of linear algebra

    3Blue1Brown

  4. 16 min
    Step 4

    Singular value decomposition

    Any matrix factors into rotate or reflect, scale, then rotate or reflect.

    SVD Visualized, Singular Value Decomposition explained | SEE Matrix , Chapter 3 #SoME2

    Visual Kernel

    fastai/numerical-linear-algebra★ 11k
  5. 10 min
    Step 5

    The multivariable chain rule

    The rule backpropagation applies over and over.

    Multivariable chain rule

    Khan Academy

  6. 6 min
    Step 6

    The Jacobian matrix

    All partial derivatives of a vector function in one matrix.

    The Jacobian matrix

    Khan Academy

  7. 6 min
    Step 7

    The Hessian matrix

    Second derivatives describe curvature around a point.

    The Hessian matrix | Multivariable calculus | Khan Academy

    Khan Academy

  8. 22 min
    Step 8

    Taylor series

    Approximate a function near a point with polynomials.

    Taylor series | Chapter 11, Essence of calculus

    3Blue1Brown

  9. 15 min
    Step 9

    Convexity

    Why every local minimum of a convex function is also a global one.

    Convexity 101 [Optimization Bootcamp]

    Steve Brunton

    cvxpy/cvxpy★ 6.4k
  10. 13 min
    Step 10

    Lagrange multipliers

    Optimize a function while obeying a constraint.

    Understanding Lagrange Multipliers Visually

    Serpentine Integral

  11. 11 min
    Step 11

    The covariance matrix

    How features vary together, summed up in one matrix.

    The Covariance Matrix : Data Science Basics

    ritvikmath

  12. 7 min
    Step 12

    The multivariate Gaussian

    The bell curve in many dimensions, shaped by covariance.

    Multivariate Normal (Gaussian) Distribution Explained

    DataMListic

  13. 31 min
    Step 13

    The central limit theorem

    Why sums of random things look normal.

    But what is the Central Limit Theorem?

    3Blue1Brown

  14. 11 min
    Step 14

    Entropy, cross-entropy, and KL divergence

    The information theory behind the most common loss function.

    A Short Introduction to Entropy, Cross-Entropy and KL-Divergence

    Aurélien Géron

Build it

Watching isn't learning. Pick one and put it on GitHub.

Next roadmap