3D Vision, NeRFs & Gaussian Splatting
Turn photos into 3D: camera geometry, stereo, structure from motion, depth networks, neural radiance fields, Gaussian splatting, and feed-forward reconstruction.
Before this: Computer Vision. Python & Math for AI (linear algebra). Deep Learning for the NeRF and splatting code. Training steps want an NVIDIA GPU or a cloud GPU.
-
21 min
Step 1Pinhole cameras and perspective projection
How a 3D point lands on a 2D image, the base of everything after.
Pinhole and Perspective Projection | Image FormationFirst Principles of Computer Vision
-
4 min
Step 2Camera intrinsics and extrinsics
The two matrices that describe a camera's lens and its pose.
Intrinsic and Extrinsic Matrices | Camera CalibrationFirst Principles of Computer Vision
-
17 min
Step 3Depth from stereo
Two calibrated cameras plus disparity give you depth by triangulation.
Simple Stereo | Camera CalibrationFirst Principles of Computer Vision
-
15 min
Step 4Epipolar geometry
The constraint linking matching points across two uncalibrated views.
Epipolar Geometry | Uncalibrated StereoFirst Principles of Computer Vision
-
57 min
Step 5Structure from motion with COLMAP
Recover camera poses and sparse 3D points from a photo collection.
Understanding 3D Reconstruction with COLMAPEveryPoint
colmap/colmap★ 13.3k -
13 min
Step 6Depth from a single image
How neural networks learn to guess depth from one photo.
How Neural Nets estimate depth from 2D images? Monocular Depth Estimation Explained!Neural Breakdown with AVB
DepthAnything/Depth-Anything-V2★ 9.2k -
1h 12m
Step 7Deep learning on 3D data
Stanford CS231n lecture on voxels, point clouds, meshes, and implicit shapes.
Stanford CS231N Deep Learning for Computer Vision | Spring 2025 | Lecture 15: 3D VisionStanford Online
-
14 min
Step 8Neural radiance fields (NeRF)
A network that maps position and view direction to color and density.
NERFs (No, not that kind) - Neural Radiance Fields - ComputerphileComputerphile
-
12 min
Step 9Code NeRF in 100 lines of PyTorch
Ray sampling, volume rendering, and training, written out in short code.
Neural Radiance Fields | NeRF in 100 lines of PyTorch codePapers in 100 Lines of Code
MaximeVandegar/Papers-in-100-Lines-of-Code★ 2.9k -
14 min
Step 10Train a NeRF on your own scene
Capture, train, and export a mesh with the open-source Nerfstudio toolkit.
How to Make 3D Models from NeRFs using NerfstudioEveryPoint
nerfstudio-project/nerfstudio★ 12.2k -
18 min
Step 113D Gaussian splatting
Swap the network for millions of 3D Gaussians that rasterize in real time.
3D Gaussian Splatting! - ComputerphileComputerphile
-
35 min
Step 12Code Gaussian splatting in PyTorch
Projection, sorting, and alpha blending of Gaussians in about 300 lines.
3D Gaussian Splatting | 3DGS Implementation from Scratch in PyTorch-OnlyPapers in 100 Lines of Code
nerfstudio-project/gsplat★ 5.9k -
8 min
Step 13From your video to a splat with free tools
A practical capture-to-viewer pipeline using RealityScan, Brush, and SuperSplat.
The Completely Free Video to 3D Gaussian Splatting PipelineJake Wong
ArthurBrussee/brush★ 5.3k -
54 min
Step 14Feed-forward reconstruction: DUSt3R and MASt3R
Talk by a co-author: one network predicts 3D points from image pairs.
Vincent Leroy: From CroCo to MASt3R: A Paradigm Change in 3D VisionMontreal Robotics
naver/mast3r★ 3.2k -
16 min
Step 15Generating 3D models with diffusion
How 2D diffusion models are used to create new 3D objects.
Generating 3D Models with Diffusion - ComputerphileComputerphile
Build it
Watching isn't learning. Pick one and put it on GitHub.
- Calibrate your phone camera with a printed checkerboard in OpenCV and report its focal length and distortion.
- Photograph one object from 40 angles, reconstruct it with COLMAP, then train both a NeRF and a Gaussian splat and compare quality and render speed.
- Run Depth Anything V2 on 20 of your photos and turn three of them into point clouds you can rotate in a viewer.