AI Roadmaps
← All roadmaps
AdvancedRoadmap 34 · 13 steps · 8h 00m of video

Self-Driving Cars & Autonomous Systems

Follow the self-driving stack from sensors and localization through perception, prediction, and planning to end-to-end models and simulation.

Before this: Deep Learning and Computer Vision. Robotics & Embodied AI helps for the planning and imitation steps.

  1. 25 min
    Step 1

    Introduction to self-driving

    Waabi's overview of the problem and the classic autonomy stack.

    [01] Introduction to Self-Driving (Waabi CVPR 24 Tutorial on Self-Driving Cars)

    Waabi

  2. 28 min
    Step 2

    Sensors and hardware

    Cameras, lidar, radar, and compute, and what each sees.

    [02] Hardware and Sensors (Waabi CVPR 24 Tutorial on Self-Driving Cars)

    Waabi

  3. 11 min
    Step 3

    Kalman filters

    Fuse noisy measurements with a motion model to track state.

    Visually Explained: Kalman Filters

    Visually Explained

  4. 27 min
    Step 4

    Localization

    Work out where the car is on the map, to centimeters.

    Localization (Waabi CVPR 23 Tutorial on Self-Driving Cars)

    Waabi

  5. 36 min
    Step 5

    HD maps

    What goes into driving maps and how they are built.

    [10] Mapping (Waabi CVPR 24 Tutorial on Self-Driving Cars)

    Waabi

  6. 26 min
    Step 6

    Perception

    Detect and track cars and pedestrians in 3D from sensor data.

    Perception (Waabi CVPR 23 Tutorial on Self-Driving Cars)

    Waabi

  7. 57 min
    Step 7

    Bird's-eye-view 3D perception

    MIT talk on efficient 3D perception, including camera and lidar BEV fusion.

    Efficient 3D Perception for Autonomous Vehicles Zhijian Liu MIT

    xLAB for Safe Autonomous Systems

  8. 54 min
    Step 8

    Motion forecasting

    Predict where other road users will move next.

    Motion Forecasting (Waabi CVPR 23 Tutorial on Self-Driving Cars)

    Waabi

  9. 52 min
    Step 9

    Motion planning and control

    Choose a safe trajectory, then follow it with steering and throttle.

    Motion Planning and Control (Waabi CVPR 23 Tutorial on Self-Driving Cars)

    Waabi

  10. 28 min
    Step 10

    Imitation learning for driving

    Tübingen lecture on learning to drive from human demonstrations.

    Self-Driving Cars - Lecture 2.3 (Imitation Learning: Imitation Learning)

    Tübingen Machine Learning

  11. 42 min
    Step 11

    End-to-end driving

    Wayve's CEO on replacing the hand-built stack with one learned model.

    How End-to-End Learning Created Autonomous Driving 2.0: Wayve CEO Alex Kendall

    Sequoia Capital

  12. 53 min
    Step 12

    Simulation

    Test driving software in simulated worlds before it reaches the road.

    [08] Simulation (Waabi CVPR 24 Tutorial on Self-Driving Cars)

    Waabi

    carla-simulator/carla★ 14.7k
  13. 41 min
    Step 13

    How openpilot controls a real car

    comma's engineer on sending steering and speed commands to stock cars.

    How Do We Control The Car? | COMMA_CON

    comma

    commaai/openpilot★ 64.2k

Build it

Watching isn't learning. Pick one and put it on GitHub.

Browse all roadmaps →