Home
GitHub Star

Track

Learn NLP

Goal: Word vectors through BERT.

Prereqs: Python plus a first ML course.

Status: done

Work through the steps in order. Bold links open YouTube.

Step Concept YouTube Read
1 Word vectors CS224N 2024 L1 — Intro and Word Vectors SLP3 ch. 5 Embeddings
2 Word2Vec StatQuest — Word Embedding and Word2Vec Word2Vec paper
3 Tokens and BPE Karpathy — Let’s build the GPT Tokenizer SLP3 ch. 2 Words and Tokens
4 Language models Karpathy — building makemore SLP3 ch. 3 N-gram Language Models
5 Neural classifiers CS224N 2024 L2 — Word Vectors and Language Models SLP3 ch. 4 Logistic Regression and Text Classification
6 Dependency parsing CS224N 2024 L4 — Dependency Parsing  
7 RNNs StatQuest — Recurrent Neural Networks  
8 LSTM StatQuest — LSTM, Clearly Explained  
9 Seq2seq StatQuest — Encoder-Decoder seq2seq  
10 Transformers (shape) 3Blue1Brown — Transformers, the tech behind LLMs SLP3 ch. 7 Transformers and Pretraining
11 Attention 3Blue1Brown — Attention in transformers, step-by-step The Illustrated Transformer
12 The Transformer paper   Attention Is All You Need
13 Build a GPT Karpathy — Let’s build GPT  
14 BERT StatQuest — Encoder-Only Transformers (BERT) BERT paper

Found a broken link or an unclear step? Report a problem with this track.