AI Roadmaps
← All roadmaps
IntermediateRoadmap 38 · 15 steps · 4h 18m of video

Classic NLP Before LLMs

The NLP toolkit that came before transformers: tokenizing and normalizing text, TF-IDF, n-grams, Naive Bayes, HMM tagging, NER, topic models, word vectors, and seq2seq with attention.

Before this: Machine Learning. Deep Learning for the seq2seq and attention steps.

  1. 10 min
    Step 1

    What NLP is

    The main NLP tasks and how a text pipeline fits together.

    What is NLP (Natural Language Processing)?

    IBM Technology

  2. 8 min
    Step 2

    Text tokenization

    Split raw text into tokens and see why the choices matter.

    NLP Demystified 2: Text Tokenization

    Future Mojo

  3. 17 min
    Step 3

    Stemming and lemmatization

    Reduce words to a shared base form with NLTK and spaCy.

    Stemming and Lemmatization: NLP Tutorial For Beginners - S1 E10

    codebasics

    codebasics/nlp-tutorials★ 341
  4. 9 min
    Step 4

    TF-IDF

    Weight word counts by how rare each word is across documents.

    Term Frequency Inverse Document Frequency (TF-IDF) Explained

    DataMListic

  5. 11 min
    Step 5

    N-gram language models

    Predict the next word from counts of the previous few words.

    NLP: Understanding the N-gram language models

    Machine Learning TV

  6. 15 min
    Step 6

    Naive Bayes for spam filtering

    A fast text classifier that assumes words are independent given the class.

    Naive Bayes, Clearly Explained!!!

    StatQuest with Josh Starmer

  7. 45 min
    Step 7

    Sentiment analysis: VADER to RoBERTa

    Score reviews with a word-list method, then a pretrained model, and compare.

    Python Sentiment Analysis Project with NLTK and 🤗 Transformers. Classify Amazon Reviews!!

    Rob Mulla

  8. 11 min
    Step 8

    Part-of-speech tagging

    Tag each word's grammatical role with a hidden Markov model.

    Part of Speech Tagging : Natural Language Processing

    ritvikmath

  9. 21 min
    Step 9

    The Viterbi algorithm

    Find the most likely tag sequence with dynamic programming.

    The Viterbi Algorithm : Natural Language Processing

    ritvikmath

  10. 23 min
    Step 10

    Named entity recognition

    Pull people, places, and organizations out of text with spaCy.

    Named Entity Recognition (NER): NLP Tutorial For Beginners - S1 E12

    codebasics

    explosion/spaCy★ 34.1k
  11. 27 min
    Step 11

    Topic modeling with LDA

    Model each document as a mix of topics, and each topic as a mix of words.

    Latent Dirichlet Allocation (Part 1 of 2)

    Luis Serrano Academy

    piskvorky/gensim★ 16.5k
  12. 16 min
    Step 12

    Word embeddings and word2vec

    Learn word vectors by predicting the words around each word.

    Word Embedding and Word2Vec, Clearly Explained!!!

    StatQuest with Josh Starmer

  13. 13 min
    Step 13

    GloVe and fastText

    Word vectors from co-occurrence counts and from subword pieces.

    Word2Vec, GloVe, FastText- EXPLAINED!

    CodeEmporium

  14. 17 min
    Step 14

    Seq2seq encoder-decoder models

    Map one sequence to another, as in translation, with two LSTMs.

    Sequence-to-Sequence (seq2seq) Encoder-Decoder Neural Networks, Clearly Explained!!!

    StatQuest with Josh Starmer

    bentrevett/pytorch-seq2seq★ 5.7k
  15. 16 min
    Step 15

    Attention for seq2seq

    Let the decoder look back at every input word, the step toward transformers.

    Attention for Neural Networks, Clearly Explained!!!

    StatQuest with Josh Starmer

Build it

Watching isn't learning. Pick one and put it on GitHub.

Browse all roadmaps →