AI Roadmaps
← All roadmaps
IntermediateRoadmap 40 · 10 steps · 3h 35m of video

Document AI & OCR

Get text, tables, and fields out of scans and PDFs: image cleanup, classic OCR engines, handwriting recognition, layout-aware models, PDF-to-Markdown tools, and vision-language models that read pages directly.

Before this: Machine Learning and Deep Learning. Computer Vision helps for the handwriting and layout steps. The last two steps run 3B-7B vision-language models, so use a GPU or Colab there.

  1. 14 min
    Step 1

    What OCR is

    How software goes from scanned pixels to recognized characters.

    Optical Character Recognition (OCR) - Computerphile

    Computerphile

  2. 53 min
    Step 2

    Clean up images before OCR

    Binarize, denoise, deskew, and remove borders with OpenCV so OCR misreads less.

    How to Preprocess Images for Text OCR in Python (OCR in Python Tutorials 02.02)

    Python Tutorials for Digital Humanities

    wjbmattingly/ocr_python_textbook★ 270
  3. 22 min
    Step 3

    Read text with pytesseract, EasyOCR, and keras-ocr

    Run three open OCR libraries on real photos and compare their results.

    Detect Text in Images with Python - pytesseract vs. easyocr vs keras_ocr

    Rob Mulla

    JaidedAI/EasyOCR★ 30.2k
  4. 22 min
    Step 4

    Handwriting recognition with a CNN, LSTM, and CTC

    Train a CNN-LSTM with CTC loss to read handwritten words from IAM.

    Step-by-Step Handwriting Recognition Tutorial Using TensorFlow

    Python Lessons

  5. 32 min
    Step 5

    Extract tables from PDFs

    Compare Python libraries for pulling tables out of PDFs, and where each fails.

    Python Libraries to Extract Tables from PDFs

    NeuralNine

  6. 29 min
    Step 6

    Layout-aware models with LayoutLM

    Feed word positions with the text so a model can label form fields.

    Extract Key Information from Documents using LayoutLM | LayoutLM Fine-tuning | Deep Learning

    Karndeep Singh

    microsoft/unilm★ 22.2k
  7. 8 min
    Step 7

    Parse documents with Docling

    Turn PDFs and office files into structured text and tables for LLM apps.

    What Is Docling? Transforming Unstructured Data for RAG and AI

    IBM Technology

    docling-project/docling★ 70.8k
  8. 14 min
    Step 8

    Convert PDFs to Markdown with Marker

    Turn complex PDFs into Markdown that keeps headings and tables.

    Marker: This Open-Source Tool will make your PDFs LLM Ready

    Prompt Engineering

    datalab-to/marker★ 41k
  9. 12 min
    Step 9

    Vision-language models as OCR: olmOCR

    A fine-tuned 7B vision-language model reads page images into clean text.

    olmOCR - The Open OCR System

    Sam Witteveen

    allenai/olmocr★ 19.8k
  10. 7 min
    Step 10

    DeepSeek-OCR and text as vision tokens

    Encode a page as a few vision tokens, then decode its text from them.

    DeepSeek-OCR Explained

    Caleb Writes Code

    deepseek-ai/DeepSeek-OCR★ 24.2k

Build it

Watching isn't learning. Pick one and put it on GitHub.

Browse all roadmaps →