Core concepts#

A few ideas come back on every page of this documentation: predictors, the Document tree they return, and the way docTR expresses positions. Ten minutes here will make everything else easier to follow.

The OCR pipeline#

OCR is split into two consecutive tasks, each solved by a dedicated deep learning model:

Text detection localizes every word on the page (where is text?).
Text recognition transcribes each detected word from its crop (what does it say?).

The predictions are then assembled into a structured document. Optional stages plug into the same pipeline when you need them: orientation handling for rotated pages, layout analysis, and table structure recognition.

Predictors and models#

You rarely call a model directly. Instead you use a predictor: a callable that takes care of everything around the neural network so you can feed it raw pages.

Layer	Responsibility	Example
Pre-processor	Resizes, pads, normalizes and batches the input images	`PreProcessor`
Model	A regular PyTorch `nn.Module` that runs the network	`fast_base`, `crnn_vgg16_bn`
Post-processor	Turns raw network outputs into boxes, strings and scores	decoding of the segmentation map

docTR exposes a factory function for every task. Each one accepts an architecture name (or a model instance, e.g. one you trained yourself) and returns a ready-to-use predictor:

Task	Factory	Returns
End-to-end OCR	`ocr_predictor()`	a `Document`
Key information extraction	`kie_predictor()`	a `KIEDocument`
Text detection only	`detection_predictor()`	boxes per page
Text recognition only	`recognition_predictor()`	`(text, confidence)` per crop
Layout analysis	`layout_predictor()`	labelled regions per page
Table structure recognition	`table_predictor()`	cells with row/column spans per table

The end-to-end ocr_predictor() composes the task predictors, which stay reachable as attributes:

from doctr.models import ocr_predictor

predictor = ocr_predictor(pretrained=True)

predictor.det_predictor  # text detection (pre-processor + model + post-processor)
predictor.reco_predictor  # text recognition
predictor.doc_builder  # assembles words into lines, blocks and pages

Important

Always pass pretrained=True unless you load your own weights: without it, the models are randomly initialized and return meaningless predictions.

The `Document` tree#

The OCR predictor returns a Document: a tree of elements that mirrors the structure of what was read.

Document └── pages: list[Page] # one per input page ├── blocks: list[Block] # groups of lines (paragraphs, addresses...) │ ├── lines: list[Line] # words read together │ │ └── words: list[Word] # value, confidence, geometry │ └── artefacts: list[Artefact] # non-textual elements ├── layout: list[LayoutElement] # with detect_layout=True └── tables: list[Table] # with detect_tables=True └── cells: list[TableCell]

Element	What it represents
`Word`	An uninterrupted sequence of characters, with its text `value`, recognition `confidence` and `geometry`
`Line`	Words aligned and meant to be read together. On a two-column page, a horizontal row holds two lines
`Block`	A group of lines (e.g. an address written on several lines) and artefacts
`Artefact`	A non-textual element: a picture, a logo, a QR code, a signature…
`LayoutElement`	A region found by the layout model: `Title`, `Text`, `Table`, `Picture`, `Page-header`…
`Table` / `TableCell`	A recognized table, as cells spanning rows and columns
`Page`	Everything found on one page, plus its `dimensions`, `orientation` and `language`

Every element can be rendered to text with .render() and exported to a JSON-serializable dict with .export(). See Working with results for a hands-on tour.

Coordinates#

All geometries are expressed in relative coordinates, between 0 and 1, as a fraction of the page width (x) and height (y), with the origin at the top-left corner. This makes them independent of the resolution the page was rendered at.

Straight boxes (default)

Two points: the top-left and the bottom-right corners.

((xmin, ymin), (xmax, ymax))
# e.g. ((0.12, 0.05), (0.31, 0.08))

Rotated boxes

With assume_straight_pages=False: a polygon of four (x, y) points, as a NumPy array of shape (4, 2).

array([[x1, y1], [x2, y2], [x3, y3], [x4, y4]])

To get pixel coordinates, multiply by the page size. Note that page.dimensions is (height, width):

height, width = page.dimensions
(xmin, ymin), (xmax, ymax) = word.geometry
box_in_pixels = (int(xmin * width), int(ymin * height), int(xmax * width), int(ymax * height))

Scores#

docTR attaches several scores to its predictions. They answer different questions:

Word.confidence: How sure the recognition model is about the text, between 0 and 1. By default it is the lowest probability among the characters of the word (see confidence aggregation).
Word.objectness_score: How sure the detection model is that the box contains text.
Word.crop_orientation: The predicted rotation of the word crop, {"value": angle, "confidence": score}, when crop orientation is classified (rotated documents).
Page.orientation / Page.language: The estimated page rotation and language, filled with detect_orientation=True / detect_language=True.

Vocabularies#

A recognition model can only output characters from the vocabulary it was trained on. The pretrained recognition models use the french vocabulary, which covers Latin characters with the most common accents, digits and punctuation. docTR ships 100+ vocabularies for training models on other languages and alphabets, and lets you restrict a model to a subset of its vocabulary.

from doctr.models import recognition_predictor

predictor = recognition_predictor("crnn_vgg16_bn", pretrained=True)
print(predictor.model.cfg["vocab"])

Next steps#

User guide

Every option of the OCR predictor, explained with examples.