AI Roadmaps
← All roadmaps
AdvancedRoadmap 36 · 15 steps · 6h 08m of video

3D Vision, NeRFs & Gaussian Splatting

Turn photos into 3D: camera geometry, stereo, structure from motion, depth networks, neural radiance fields, Gaussian splatting, and feed-forward reconstruction.

Before this: Computer Vision. Python & Math for AI (linear algebra). Deep Learning for the NeRF and splatting code. Training steps want an NVIDIA GPU or a cloud GPU.

  1. 21 min
    Step 1

    Pinhole cameras and perspective projection

    How a 3D point lands on a 2D image, the base of everything after.

    Pinhole and Perspective Projection | Image Formation

    First Principles of Computer Vision

  2. 4 min
    Step 2

    Camera intrinsics and extrinsics

    The two matrices that describe a camera's lens and its pose.

    Intrinsic and Extrinsic Matrices | Camera Calibration

    First Principles of Computer Vision

  3. 17 min
    Step 3

    Depth from stereo

    Two calibrated cameras plus disparity give you depth by triangulation.

    Simple Stereo | Camera Calibration

    First Principles of Computer Vision

  4. 15 min
    Step 4

    Epipolar geometry

    The constraint linking matching points across two uncalibrated views.

    Epipolar Geometry | Uncalibrated Stereo

    First Principles of Computer Vision

  5. 57 min
    Step 5

    Structure from motion with COLMAP

    Recover camera poses and sparse 3D points from a photo collection.

    Understanding 3D Reconstruction with COLMAP

    EveryPoint

    colmap/colmap★ 13.3k
  6. 13 min
    Step 6

    Depth from a single image

    How neural networks learn to guess depth from one photo.

    How Neural Nets estimate depth from 2D images? Monocular Depth Estimation Explained!

    Neural Breakdown with AVB

    DepthAnything/Depth-Anything-V2★ 9.2k
  7. 1h 12m
    Step 7

    Deep learning on 3D data

    Stanford CS231n lecture on voxels, point clouds, meshes, and implicit shapes.

    Stanford CS231N Deep Learning for Computer Vision | Spring 2025 | Lecture 15: 3D Vision

    Stanford Online

  8. 14 min
    Step 8

    Neural radiance fields (NeRF)

    A network that maps position and view direction to color and density.

    NERFs (No, not that kind) - Neural Radiance Fields - Computerphile

    Computerphile

  9. 12 min
    Step 9

    Code NeRF in 100 lines of PyTorch

    Ray sampling, volume rendering, and training, written out in short code.

    Neural Radiance Fields | NeRF in 100 lines of PyTorch code

    Papers in 100 Lines of Code

    MaximeVandegar/Papers-in-100-Lines-of-Code★ 2.9k
  10. 14 min
    Step 10

    Train a NeRF on your own scene

    Capture, train, and export a mesh with the open-source Nerfstudio toolkit.

    How to Make 3D Models from NeRFs using Nerfstudio

    EveryPoint

    nerfstudio-project/nerfstudio★ 12.2k
  11. 18 min
    Step 11

    3D Gaussian splatting

    Swap the network for millions of 3D Gaussians that rasterize in real time.

    3D Gaussian Splatting! - Computerphile

    Computerphile

  12. 35 min
    Step 12

    Code Gaussian splatting in PyTorch

    Projection, sorting, and alpha blending of Gaussians in about 300 lines.

    3D Gaussian Splatting | 3DGS Implementation from Scratch in PyTorch-Only

    Papers in 100 Lines of Code

    nerfstudio-project/gsplat★ 5.9k
  13. 8 min
    Step 13

    From your video to a splat with free tools

    A practical capture-to-viewer pipeline using RealityScan, Brush, and SuperSplat.

    The Completely Free Video to 3D Gaussian Splatting Pipeline

    Jake Wong

    ArthurBrussee/brush★ 5.3k
  14. 54 min
    Step 14

    Feed-forward reconstruction: DUSt3R and MASt3R

    Talk by a co-author: one network predicts 3D points from image pairs.

    Vincent Leroy: From CroCo to MASt3R: A Paradigm Change in 3D Vision

    Montreal Robotics

    naver/mast3r★ 3.2k
  15. 16 min
    Step 15

    Generating 3D models with diffusion

    How 2D diffusion models are used to create new 3D objects.

    Generating 3D Models with Diffusion - Computerphile

    Computerphile

Build it

Watching isn't learning. Pick one and put it on GitHub.

Browse all roadmaps →