Document AI & OCR
Get text, tables, and fields out of scans and PDFs: image cleanup, classic OCR engines, handwriting recognition, layout-aware models, PDF-to-Markdown tools, and vision-language models that read pages directly.
Before this: Machine Learning and Deep Learning. Computer Vision helps for the handwriting and layout steps. The last two steps run 3B-7B vision-language models, so use a GPU or Colab there.
-
14 min
Step 1What OCR is
How software goes from scanned pixels to recognized characters.
Optical Character Recognition (OCR) - ComputerphileComputerphile
-
53 min
Step 2Clean up images before OCR
Binarize, denoise, deskew, and remove borders with OpenCV so OCR misreads less.
How to Preprocess Images for Text OCR in Python (OCR in Python Tutorials 02.02)Python Tutorials for Digital Humanities
wjbmattingly/ocr_python_textbook★ 270 -
22 min
Step 3Read text with pytesseract, EasyOCR, and keras-ocr
Run three open OCR libraries on real photos and compare their results.
Detect Text in Images with Python - pytesseract vs. easyocr vs keras_ocrRob Mulla
JaidedAI/EasyOCR★ 30.2k -
22 min
Step 4Handwriting recognition with a CNN, LSTM, and CTC
Train a CNN-LSTM with CTC loss to read handwritten words from IAM.
Step-by-Step Handwriting Recognition Tutorial Using TensorFlowPython Lessons
-
32 min
Step 5Extract tables from PDFs
Compare Python libraries for pulling tables out of PDFs, and where each fails.
Python Libraries to Extract Tables from PDFsNeuralNine
-
29 min
Step 6Layout-aware models with LayoutLM
Feed word positions with the text so a model can label form fields.
Extract Key Information from Documents using LayoutLM | LayoutLM Fine-tuning | Deep LearningKarndeep Singh
microsoft/unilm★ 22.2k -
8 min
Step 7Parse documents with Docling
Turn PDFs and office files into structured text and tables for LLM apps.
What Is Docling? Transforming Unstructured Data for RAG and AIIBM Technology
docling-project/docling★ 70.8k -
14 min
Step 8Convert PDFs to Markdown with Marker
Turn complex PDFs into Markdown that keeps headings and tables.
Marker: This Open-Source Tool will make your PDFs LLM ReadyPrompt Engineering
datalab-to/marker★ 41k -
12 min
Step 9Vision-language models as OCR: olmOCR
A fine-tuned 7B vision-language model reads page images into clean text.
olmOCR - The Open OCR SystemSam Witteveen
allenai/olmocr★ 19.8k -
7 min
Step 10DeepSeek-OCR and text as vision tokens
Encode a page as a few vision tokens, then decode its text from them.
DeepSeek-OCR ExplainedCaleb Writes Code
deepseek-ai/DeepSeek-OCR★ 24.2k
Build it
Watching isn't learning. Pick one and put it on GitHub.
- Photograph 20 receipts, clean them with OpenCV, run two OCR engines, and report each one's character error rate against your own transcriptions.
- Convert 10 PDFs with tables into Markdown using Docling and Marker, and write up where each one broke a table or the reading order.
- Fine-tune LayoutLM on the FUNSD forms dataset to label questions and answers, and report per-label F1.