doctr.io#
Read documents into pages, represent the predictions as a tree of elements, and export them.
Reading files#
- class DocumentFile[source]#
Read a document from multiple extensions
- classmethod from_pdf(file, **kwargs)[source]#
Read a PDF file
>>> from doctr.io import DocumentFile >>> doc = DocumentFile.from_pdf("path/to/your/doc.pdf")
- classmethod from_images(files, **kwargs)[source]#
Read an image file (or a collection of image files) and convert it into an image in numpy format
>>> from doctr.io import DocumentFile >>> pages = DocumentFile.from_images(["path/to/your/page1.png", "path/to/your/page2.png"])
- Parameters:
- Returns:
the list of pages decoded as numpy ndarray of shape H x W x 3
- Return type:
- read_pdf(file, scale=2, rgb_mode=True, password=None, **kwargs)[source]#
Read a PDF file and convert it into an image in numpy format
>>> from doctr.io import read_pdf >>> doc = read_pdf("path/to/your/doc.pdf")
- Parameters:
- Returns:
the list of pages decoded as numpy ndarray of shape H x W x C
- Return type:
- read_img_as_numpy(file, output_size=None, rgb_output=True)[source]#
Read an image file into numpy format
>>> from doctr.io import read_img_as_numpy >>> page = read_img_as_numpy("path/to/your/doc.jpg")
- Parameters:
- Returns:
the page decoded as numpy ndarray of shape H x W x 3
- Return type:
- decode_img_as_tensor(img_content, dtype=torch.float32)[source]#
Read a byte stream as a PyTorch tensor
Document structure#
The elements returned by the predictors, from the largest to the smallest. See The Document tree for an overview.
Document#
- class Document(pages)[source]#
Implements a document element as a collection of pages
- render(page_break='\n\n\n\n', **kwargs)#
Renders the full text of the document, with the content of each page sorted in reading order.
- export(reading_order=True)#
Export the document into a nested dict, with the content of each page sorted in reading order.
- export_as(format, **kwargs)#
Export the document in the requested format (‘markdown’/’md’, ‘asciidoc’/’adoc’, ‘html’, ‘text’/’txt’, ‘json’/’dict’, ‘xml’/’hocr’).
- Return type:
- export_as_markdown(page_break='\n\n---\n\n', **kwargs)#
Export the document as Markdown, with the content of each page sorted in reading order.
- export_as_asciidoc(page_break='\n\n<<<\n\n', **kwargs)#
Export the document as AsciiDoc, with the content of each page sorted in reading order.
- export_as_html(page_break='<hr>', **kwargs)#
Export the document as semantic HTML, with the content of each page sorted in reading order.
- export_as_xml(**kwargs)#
Export the document as XML (hOCR-format)
- Parameters:
**kwargs (Any) – additional keyword arguments passed to the XML page export
- Returns:
list of tuple of (bytes, ElementTree)
- Return type:
Page#
- class Page(
- page,
- blocks,
- page_idx,
- dimensions,
- orientation=None,
- language=None,
- layout=None,
- tables=None,
Implements a page element as a collection of blocks
- Parameters:
page (ndarray) – image encoded as a numpy array in uint8
page_idx (int) – the index of the page in the input raw document
dimensions (tuple[int, int]) – the page size in pixels in format (height, width)
orientation (dict[str, Any] | None) – a dictionary with the value of the rotation angle in degress and confidence of the prediction
language (dict[str, Any] | None) – a dictionary with the language value and confidence of the prediction
layout (list[LayoutElement]) – optional list of layout regions detected on the page
tables (list[Table]) – optional list of tables recognized on the page. Words assigned to a table are removed from blocks.
- render(block_break='\n\n', direction='auto', include_furniture=True)#
Renders the full text of the page, with its content sorted in reading order.
- Parameters:
- Returns:
the text of the page
- Return type:
- export(reading_order=True)#
Export the page into a nested dict, with its content sorted in reading order.
- export_as(format, **kwargs)#
Export the page in the requested format.
- export_as_markdown(
- direction='auto',
- escape=True,
- include_furniture=True,
- images='placeholder',
Export the page as Markdown, with its content sorted in reading order.
- Parameters:
direction (str) – reading direction, one of ‘auto’, ‘ltr’, ‘rtl’, ‘ttb-rtl’ or ‘ttb-ltr’
escape (bool) – whether the characters carrying a structural meaning in Markdown should be escaped
include_furniture (bool) – whether page headers, page footers and footnotes should be included
images (str | FigureEncoder | None) – how to render the figures: ‘none’, ‘placeholder’, ‘embedded’, or a
FigureEncoder(required for ‘referenced’)
- Returns:
a Markdown string
- Return type:
- export_as_asciidoc(
- direction='auto',
- escape=True,
- include_furniture=True,
- images='placeholder',
Export the page as AsciiDoc, with its content sorted in reading order.
- Parameters:
direction (str) – reading direction, one of ‘auto’, ‘ltr’, ‘rtl’, ‘ttb-rtl’ or ‘ttb-ltr’
escape (bool) – whether the characters and line markers carrying a structural meaning in AsciiDoc should be escaped
include_furniture (bool) – whether page headers, page footers and footnotes should be included
images (str | FigureEncoder | None) – how to render the figures: ‘none’, ‘placeholder’, ‘embedded’, or a
FigureEncoder(required for ‘referenced’)
- Returns:
an AsciiDoc string
- Return type:
- export_as_html(direction='auto', include_furniture=True, images='placeholder')#
Export the page as semantic HTML, with its content sorted in reading order.
- Parameters:
direction (str) – reading direction, one of ‘auto’, ‘ltr’, ‘rtl’, ‘ttb-rtl’ or ‘ttb-ltr’
include_furniture (bool) – whether page headers, page footers and footnotes should be included
images (str | FigureEncoder | None) – how to render the figures: ‘none’, ‘placeholder’, ‘embedded’, or a
FigureEncoder(required for ‘referenced’)
- Returns:
an HTML string
- Return type:
- export_as_xml(
- file_title='docTR - XML export (hOCR)',
- direction='auto',
- reading_order=True,
- dpi=72,
Export the page as XML (hOCR-format), with its content sorted in reading order convention: kba/hocr-spec
- Parameters:
file_title (str) – the title of the XML file
direction (str) – reading direction, one of ‘auto’, ‘ltr’, ‘rtl’, ‘ttb-rtl’ or ‘ttb-ltr’
reading_order (bool) – whether the content should be linearized in reading order
dpi (int) – the page resolution in dots per inch, used to estimate font sizes (x_size, x_fsize)
- Returns:
a tuple of the XML byte string, and its ElementTree
- Return type:
- items_in_reading_order(direction='auto', include_figures=False)#
Return the content of the page (blocks, tables and optionally figures) sorted in reading order.
Block, Line and Word#
- class Block(lines=None, artefacts=None, geometry=None, objectness_score=None)[source]#
Implements a block element as a collection of lines and artefacts
- Parameters:
geometry (tuple[tuple[float, float], tuple[float, float]] | ndarray | None) – bounding box of the word in format ((xmin, ymin), (xmax, ymax)) where coordinates are relative to the page’s size. If not specified, it will be resolved by default to the smallest bounding box enclosing all lines and artefacts in it.
- class Line(words, geometry=None, objectness_score=None)[source]#
Implements a line element as a collection of words
- Parameters:
- class Word(value, confidence, geometry, objectness_score, crop_orientation)[source]#
Implements a word element
- Parameters:
value (str) – the text string of the word
confidence (float) – the confidence associated with the text prediction
geometry (tuple[tuple[float, float], tuple[float, float]] | ndarray) – bounding box of the word in format ((xmin, ymin), (xmax, ymax)) where coordinates are relative to the page’s size
objectness_score (float) – the objectness score of the detection
crop_orientation (dict[str, Any]) – the general orientation of the crop in degrees and its confidence
Layout and tables#
- class LayoutElement(layout_type, confidence, geometry)[source]#
Implements a layout region predicted by a layout detection model
- Parameters:
layout_type (str) – the predicted region class (e.g. ‘Title’, ‘Text’, ‘Table’, ‘Page-header’)
confidence (float) – the confidence of the region prediction
geometry (tuple[tuple[float, float], tuple[float, float]] | ndarray) – bounding box of the word in format ((xmin, ymin), (xmax, ymax)) where coordinates are relative to the page’s size
- class Table(cells, num_rows, num_cols, geometry, confidence=1.0)[source]#
Implements a table recognized on a page as a grid of cells
The recognized text of the words falling inside the table is regrouped here and removed from the regular blocks output of the page, so it is not duplicated. The structured content can be loaded directly into pandas, e.g. pd.DataFrame(table.to_grid()).
- Parameters:
cells (list[TableCell]) – list of TableCell objects composing the table
num_rows (int) – number of rows of the table
num_cols (int) – number of columns of the table
geometry (tuple[tuple[float, float], tuple[float, float]] | ndarray) – bounding box enclosing the whole table, with coordinates relative to the page’s size
confidence (float) – the confidence of the table structure prediction (mean of the objectness_score of its cells)
- to_grid()[source]#
Return the table content as a dense num_rows x num_cols grid of strings.
Cells spanning several rows/columns have their value placed at their top-left position; the remaining positions they span are left empty. The result is directly loadable into pandas via pd.DataFrame(table.to_grid()).
- class TableCell(
- value,
- confidence,
- geometry,
- row_start,
- row_end,
- col_start,
- col_end,
- objectness_score=None,
Implements a single cell of a recognized table
- Parameters:
value (str) – the text content of the cell (words assigned to the cell, joined together)
confidence (float) – the mean recognition confidence of the words assigned to the cell (its objectness_score if no word is assigned to it)
geometry (tuple[tuple[float, float], tuple[float, float]] | ndarray) – bounding box of the cell in format ((xmin, ymin), (xmax, ymax)) or a (4, 2) polygon, with coordinates relative to the page’s size
row_start (int) – index of the first row spanned by the cell (0-indexed)
row_end (int) – index of the last row spanned by the cell (0-indexed, inclusive)
col_start (int) – index of the first column spanned by the cell (0-indexed)
col_end (int) – index of the last column spanned by the cell (0-indexed, inclusive)
objectness_score (float | None) – the score of the cell predicted by the table structure model, None if unknown
Key information extraction#
Returned by kie_predictor(): predictions are grouped by class instead of by spatial layout.
- class KIEDocument(pages)[source]#
Implements a document element as a collection of pages
- show(**kwargs)#
Overlay the result on a given image
- class KIEPage(
- page,
- predictions,
- page_idx,
- dimensions,
- orientation=None,
- language=None,
- layout=None,
Implements a KIE page element as a collection of predictions
- Parameters:
predictions (dict[str, list[Prediction]]) – Dictionary with list of block elements for each detection class
page (ndarray) – image encoded as a numpy array in uint8
page_idx (int) – the index of the page in the input raw document
dimensions (tuple[int, int]) – the page size in pixels in format (height, width)
orientation (dict[str, Any] | None) – a dictionary with the value of the rotation angle in degress and confidence of the prediction
language (dict[str, Any] | None) – a dictionary with the language value and confidence of the prediction
layout (list[LayoutElement]) – optional list of layout regions detected on the page
- render(prediction_break='\n\n', direction='auto')#
Renders the full text of the page, with the predictions of each class sorted in reading order.
- export(reading_order=True)#
Export the KIE page into a nested dict, with the predictions of each class in reading order.
- export_as(format, **kwargs)#
Export the KIE page in the requested format (‘markdown’/’md’, ‘asciidoc’/’adoc’, ‘html’, ‘text’/’txt’, ‘json’/’dict’, ‘xml’/’hocr’).
- Return type:
- export_as_markdown(direction='auto', escape=True)#
Export the KIE page as Markdown, with the predictions of each class sorted in reading order.
- export_as_asciidoc(direction='auto', escape=True)#
Export the KIE page as AsciiDoc, with the predictions of each class sorted in reading order.
- export_as_html(direction='auto')#
Export the KIE page as semantic HTML, with the predictions of each class sorted in reading order
- Return type:
- export_as_xml(
- file_title='docTR - XML export (hOCR)',
- direction='auto',
- reading_order=True,
- dpi=72,
Export the page as XML (hOCR-format), with the predictions of each class in reading order convention: kba/hocr-spec
- Parameters:
file_title (str) – the title of the XML file
direction (str) – reading direction, one of ‘auto’, ‘ltr’, ‘rtl’, ‘ttb-rtl’ or ‘ttb-ltr’
reading_order (bool) – whether the predictions of each class should be sorted in reading order
dpi (int) – the page resolution in dots per inch, used to estimate font sizes (x_size, x_fsize)
- Returns:
a tuple of the XML byte string, and its ElementTree
- Return type:
Exporters#
The export_as_* methods of the elements delegate to these exporters, which share the same reading-order
linearization. See Exporting results.
- class TextExporter[source]#
Export OCR results to plain text, with the content sorted in reading order.
>>> from doctr.io import TextExporter >>> text = TextExporter().export_page(page)
- export_document(document, page_break=None, **kwargs)#
Export a document page by page.
- export_kie_page(page, direction='auto', escape=True)#
Export a KIE page, with the predictions of each class sorted in reading order.
- Parameters:
- Returns:
the exported page as a string, with one section per detection class
- Return type:
- export_page(
- page,
- direction='auto',
- escape=True,
- include_furniture=True,
- block_break=None,
- images='placeholder',
Export a page, with its content sorted in reading order.
- Parameters:
page (Page) – the page to export
direction (str) – reading direction, one of ‘auto’, ‘ltr’, ‘rtl’, ‘ttb-rtl’ or ‘ttb-ltr’
escape (bool) – whether the characters or markers carrying a structural meaning should be neutralized
include_furniture (bool) – whether page headers, page footers and footnotes should be included
block_break (str | None) – the string inserted between two blocks (the format-specific default when None)
images (str | FigureEncoder | None) – how to render the figures: ‘none’, ‘placeholder’, ‘embedded’, or a
FigureEncoder(required for ‘referenced’)
- Returns:
the exported page as a string
- Return type:
- class MarkdownExporter[source]#
Export OCR results to Markdown, with the content sorted in reading order.
>>> from doctr.io import MarkdownExporter >>> markdown = MarkdownExporter().export_page(page)
- export_document(document, page_break=None, **kwargs)#
Export a document page by page.
- export_kie_page(page, direction='auto', escape=True)#
Export a KIE page, with the predictions of each class sorted in reading order.
- Parameters:
- Returns:
the exported page as a string, with one section per detection class
- Return type:
- export_page(
- page,
- direction='auto',
- escape=True,
- include_furniture=True,
- block_break=None,
- images='placeholder',
Export a page, with its content sorted in reading order.
- Parameters:
page (Page) – the page to export
direction (str) – reading direction, one of ‘auto’, ‘ltr’, ‘rtl’, ‘ttb-rtl’ or ‘ttb-ltr’
escape (bool) – whether the characters or markers carrying a structural meaning should be neutralized
include_furniture (bool) – whether page headers, page footers and footnotes should be included
block_break (str | None) – the string inserted between two blocks (the format-specific default when None)
images (str | FigureEncoder | None) – how to render the figures: ‘none’, ‘placeholder’, ‘embedded’, or a
FigureEncoder(required for ‘referenced’)
- Returns:
the exported page as a string
- Return type:
- class AsciiDocExporter[source]#
Export OCR results to AsciiDoc, with the content sorted in reading order.
>>> from doctr.io import AsciiDocExporter >>> asciidoc = AsciiDocExporter().export_page(page)
- export_document(document, page_break=None, **kwargs)#
Export a document page by page.
- export_kie_page(page, direction='auto', escape=True)#
Export a KIE page, with the predictions of each class sorted in reading order.
- Parameters:
- Returns:
the exported page as a string, with one section per detection class
- Return type:
- export_page(
- page,
- direction='auto',
- escape=True,
- include_furniture=True,
- block_break=None,
- images='placeholder',
Export a page, with its content sorted in reading order.
- Parameters:
page (Page) – the page to export
direction (str) – reading direction, one of ‘auto’, ‘ltr’, ‘rtl’, ‘ttb-rtl’ or ‘ttb-ltr’
escape (bool) – whether the characters or markers carrying a structural meaning should be neutralized
include_furniture (bool) – whether page headers, page footers and footnotes should be included
block_break (str | None) – the string inserted between two blocks (the format-specific default when None)
images (str | FigureEncoder | None) – how to render the figures: ‘none’, ‘placeholder’, ‘embedded’, or a
FigureEncoder(required for ‘referenced’)
- Returns:
the exported page as a string
- Return type:
- class HTMLExporter[source]#
Export OCR results to semantic HTML, with the content sorted in reading order.
Headings map to <h1>/<h2>, list items to <ul><li>, recognized tables to <table> and paragraphs to <p> (with <br> between the visual lines of a paragraph). The output is a fragment, not a full document: it carries no doctype, <html> or charset declaration.
Warning
The recognized text is HTML-escaped by default. Passing
escape=Falseinterpolates the OCR output into the markup verbatim, so a document containing markup yields active HTML. Only disable escaping for output that is never rendered in a browser.>>> from doctr.io import HTMLExporter >>> html = HTMLExporter().export_page(page)
- export_kie_page(page, direction='auto', escape=True)[source]#
Export a KIE page, with the predictions of each class sorted in reading order.
- Parameters:
- Returns:
the exported page as a string, with one section per detection class
- Return type:
- export_document(document, page_break=None, **kwargs)#
Export a document page by page.
- export_page(
- page,
- direction='auto',
- escape=True,
- include_furniture=True,
- block_break=None,
- images='placeholder',
Export a page, with its content sorted in reading order.
- Parameters:
page (Page) – the page to export
direction (str) – reading direction, one of ‘auto’, ‘ltr’, ‘rtl’, ‘ttb-rtl’ or ‘ttb-ltr’
escape (bool) – whether the characters or markers carrying a structural meaning should be neutralized
include_furniture (bool) – whether page headers, page footers and footnotes should be included
block_break (str | None) – the string inserted between two blocks (the format-specific default when None)
images (str | FigureEncoder | None) – how to render the figures: ‘none’, ‘placeholder’, ‘embedded’, or a
FigureEncoder(required for ‘referenced’)
- Returns:
the exported page as a string
- Return type:
- class XMLExporter[source]#
hOCR (XML) exporter for pages, KIE pages and documents. See the hOCR 1.2 specification for the XML convention: kba/hocr-spec
>>> from doctr.io import XMLExporter >>> xml_bytes, xml_tree = XMLExporter().export_page(page)
- export_page(
- page,
- file_title='docTR - XML export (hOCR)',
- direction='auto',
- reading_order=True,
- dpi=72,
Export a page as hOCR XML, with its content sorted in reading order.
Figures are exported as ocr_photo areas.
- Parameters:
page (Page) – the page to export
file_title (str) – the title of the XML file
direction (str) – reading direction, one of ‘auto’, ‘ltr’, ‘rtl’, ‘ttb-rtl’ or ‘ttb-ltr’
reading_order (bool) – whether the content should be linearized in reading order. Pass False to serialize page.blocks then page.tables in their raw order.
dpi (int) – the page resolution in dots per inch, used to estimate font sizes (x_size, x_fsize)
- Returns:
a tuple of the XML byte string, and its ElementTree
- Return type:
- export_kie_page(
- page,
- file_title='docTR - XML export (hOCR)',
- direction='auto',
- reading_order=True,
- dpi=72,
Export a KIE page as hOCR XML, with the predictions of each class sorted in reading order.
- Parameters:
page (KIEPage) – the KIE page to export
file_title (str) – the title of the XML file
direction (str) – reading direction, one of ‘auto’, ‘ltr’, ‘rtl’, ‘ttb-rtl’ or ‘ttb-ltr’
reading_order (bool) – whether the predictions of each class should be sorted in reading order
dpi (int) – the page resolution in dots per inch, used to estimate font sizes (x_size, x_fsize)
- Returns:
a tuple of the XML byte string, and its ElementTree
- Return type:
- page_reading_order(page, direction='auto', include_figures=False)[source]#
Linearize the content of a page (blocks, tables and optionally figures) in reading order
The result is cached on the page, so a page exported to several formats is ordered once. The figures always take part in the ordering, include_figures only controls whether they are returned.
- Parameters:
- Returns:
the ordered items, their layout label (None without layout) and the effective reading direction
- Return type:
Figures#
Rendering of the figures found by the layout model in the Markdown, AsciiDoc and HTML exports. See Figures.
- class FigureEncoder(
- mode='placeholder',
- image_dir=None,
- path_prefix=None,
- image_format='png',
- quality=95,
- padding=0.0,
Render the figures detected by the layout model in the Markdown, AsciiDoc and HTML exports
none: no figuresplaceholder: a comment marks each figureembedded: each crop is inlined as a base64 data URIreferenced: each crop is written toimage_dirand linked by its relative path
With their pixels (‘embedded’ or ‘referenced’), the text recognized inside the figures is left out of the export. Written files are named after the figure position and content, e.g.
page1_figure2-3fa2b1c9.png.>>> from doctr.io import FigureEncoder >>> markdown = page.export_as_markdown(images=FigureEncoder("referenced", image_dir="assets"))
- Parameters:
mode (str) – one of ‘none’, ‘placeholder’, ‘embedded’ or ‘referenced’
image_dir (str | Path | None) – directory the crops are written to (required in ‘referenced’ mode)
path_prefix (str | os.PathLike[str] | None) – location of
image_dirrelative to the export (image_dirby default)image_format (str) – one of ‘png’, ‘jpg’, ‘jpeg’ or ‘webp’
quality (int) – quality of the lossy formats, between 0 and 100
padding (float) – relative margin added around each region
- classmethod resolve(images)[source]#
Build an encoder from the images argument of an export method
- Parameters:
images (str | FigureEncoder | None) – an image mode, an encoder, or None (same as ‘none’)
- Returns:
the encoder
- Return type:
- source(page, region, index)[source]#
Return the image source of a figure, encoded (and written) on first use
- Parameters:
page (Page) – the page of the figure
region (LayoutElement) – the figure region
index (int) – 1-based index of the figure on the page, used in the file name
- Returns:
a data URI or a relative path, None without pixels
- Return type:
str | None
- crop_layout_region(page_img, geometry, padding=0.0)[source]#
Crop a layout region out of its page image, de-rotating rotated regions
- Parameters:
- Returns:
the crop, or None without pixels or for a region smaller than 2x2 pixels
- Return type:
ndarray | None
- encode_crop(crop, image_format='png', quality=95)[source]#
Encode an RGB(A) crop into an image file format
- picture_regions(page)[source]#
Return the figure regions of a page, without those nested in a larger figure
- Parameters:
page (Page) – the page
- Returns:
the figure regions, in detection order
- Return type:
- is_picture_region(region)[source]#
Check whether a layout region is a figure
- Parameters:
region (LayoutElement) – the layout region
- Returns:
True for a figure region
- Return type: