doctr.models#
doctr.models.classification#
- vgg16_bn_r(pretrained=False, **kwargs)[source]#
VGG-16 architecture as described in “Very Deep Convolutional Networks for Large-Scale Image Recognition”, modified by adding batch normalization, rectangular pooling and a simpler classification head.
>>> import torch >>> from doctr.models import vgg16_bn_r >>> model = vgg16_bn_r(pretrained=False) >>> input_tensor = torch.rand((1, 3, 512, 512), dtype=torch.float32) >>> out = model(input_tensor)
- resnet18(pretrained=False, **kwargs)[source]#
ResNet-18 architecture as described in “Deep Residual Learning for Image Recognition”,.
>>> import torch >>> from doctr.models import resnet18 >>> model = resnet18(pretrained=False) >>> input_tensor = torch.rand((1, 3, 512, 512), dtype=torch.float32) >>> out = model(input_tensor)
- resnet34(pretrained=False, **kwargs)[source]#
ResNet-34 architecture as described in “Deep Residual Learning for Image Recognition”,.
>>> import torch >>> from doctr.models import resnet34 >>> model = resnet34(pretrained=False) >>> input_tensor = torch.rand((1, 3, 512, 512), dtype=torch.float32) >>> out = model(input_tensor)
- resnet50(pretrained=False, **kwargs)[source]#
ResNet-50 architecture as described in “Deep Residual Learning for Image Recognition”,.
>>> import torch >>> from doctr.models import resnet50 >>> model = resnet50(pretrained=False) >>> input_tensor = torch.rand((1, 3, 512, 512), dtype=torch.float32) >>> out = model(input_tensor)
- resnet31(pretrained=False, **kwargs)[source]#
Resnet31 architecture with rectangular pooling windows as described in “Show, Attend and Read:A Simple and Strong Baseline for Irregular Text Recognition”,. Downsizing: (H, W) –> (H/8, W/4)
>>> import torch >>> from doctr.models import resnet31 >>> model = resnet31(pretrained=False) >>> input_tensor = torch.rand((1, 3, 512, 512), dtype=torch.float32) >>> out = model(input_tensor)
- mobilenet_v3_small(pretrained=False, **kwargs)[source]#
MobileNetV3-Small architecture as described in “Searching for MobileNetV3”,.
>>> import torch >>> from doctr.models import mobilenet_v3_small >>> model = mobilenetv3_small(pretrained=False) >>> input_tensor = torch.rand((1, 3, 512, 512), dtype=torch.float32) >>> out = model(input_tensor)
- mobilenet_v3_large(pretrained=False, **kwargs)[source]#
MobileNetV3-Large architecture as described in “Searching for MobileNetV3”,.
>>> import torch >>> from doctr.models import mobilenet_v3_large >>> model = mobilenet_v3_large(pretrained=False) >>> input_tensor = torch.rand((1, 3, 512, 512), dtype=torch.float32) >>> out = model(input_tensor)
- mobilenet_v3_small_r(pretrained=False, **kwargs)[source]#
MobileNetV3-Small architecture as described in “Searching for MobileNetV3”,, with rectangular pooling.
>>> import torch >>> from doctr.models import mobilenet_v3_small_r >>> model = mobilenet_v3_small_r(pretrained=False) >>> input_tensor = torch.rand((1, 3, 512, 512), dtype=torch.float32) >>> out = model(input_tensor)
- mobilenet_v3_large_r(pretrained=False, **kwargs)[source]#
MobileNetV3-Large architecture as described in “Searching for MobileNetV3”,, with rectangular pooling.
>>> import torch >>> from doctr.models import mobilenet_v3_large_r >>> model = mobilenet_v3_large_r(pretrained=False) >>> input_tensor = torch.rand((1, 3, 512, 512), dtype=torch.float32) >>> out = model(input_tensor)
- mobilenet_v3_small_crop_orientation(pretrained=False, **kwargs)[source]#
MobileNetV3-Small architecture as described in “Searching for MobileNetV3”,.
>>> import torch >>> from doctr.models import mobilenet_v3_small_crop_orientation >>> model = mobilenet_v3_small_crop_orientation(pretrained=False) >>> input_tensor = torch.rand((1, 3, 512, 512), dtype=torch.float32) >>> out = model(input_tensor)
- mobilenet_v3_small_page_orientation(pretrained=False, **kwargs)[source]#
MobileNetV3-Small architecture as described in “Searching for MobileNetV3”,.
>>> import torch >>> from doctr.models import mobilenet_v3_small_page_orientation >>> model = mobilenet_v3_small_page_orientation(pretrained=False) >>> input_tensor = torch.rand((1, 3, 512, 512), dtype=torch.float32) >>> out = model(input_tensor)
- magc_resnet31(pretrained=False, **kwargs)[source]#
Resnet31 architecture with Multi-Aspect Global Context Attention as described in “MASTER: Multi-Aspect Non-local Network for Scene Text Recognition”,.
>>> import torch >>> from doctr.models import magc_resnet31 >>> model = magc_resnet31(pretrained=False) >>> input_tensor = torch.rand((1, 3, 224, 224), dtype=tf.float32) >>> out = model(input_tensor)
- vit_s(pretrained=False, **kwargs)[source]#
VisionTransformer-S architecture “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale”,. Patches: (H, W) -> (H/8, W/8)
NOTE: unofficial config used in ViTSTR and ParSeq
>>> import torch >>> from doctr.models import vit_s >>> model = vit_s(pretrained=False) >>> input_tensor = torch.rand((1, 3, 32, 32), dtype=tf.float32) >>> out = model(input_tensor)
- vit_b(pretrained=False, **kwargs)[source]#
VisionTransformer-B architecture as described in “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale”,. Patches: (H, W) -> (H/8, W/8)
>>> import torch >>> from doctr.models import vit_b >>> model = vit_b(pretrained=False) >>> input_tensor = torch.rand((1, 3, 32, 32), dtype=tf.float32) >>> out = model(input_tensor)
- textnet_tiny(pretrained=False, **kwargs)[source]#
Implements TextNet architecture from “FAST: Faster Arbitrarily-Shaped Text Detector with Minimalist Kernel Representation”. Implementation based on the official Pytorch implementation: <czczup/FAST>`_.
>>> import torch >>> from doctr.models import textnet_tiny >>> model = textnet_tiny(pretrained=False) >>> input_tensor = torch.rand((1, 3, 512, 512), dtype=torch.float32) >>> out = model(input_tensor)
- textnet_small(pretrained=False, **kwargs)[source]#
Implements TextNet architecture from “FAST: Faster Arbitrarily-Shaped Text Detector with Minimalist Kernel Representation”. Implementation based on the official Pytorch implementation: <czczup/FAST>`_.
>>> import torch >>> from doctr.models import textnet_small >>> model = textnet_small(pretrained=False) >>> input_tensor = torch.rand((1, 3, 512, 512), dtype=torch.float32) >>> out = model(input_tensor)
- textnet_base(pretrained=False, **kwargs)[source]#
Implements TextNet architecture from “FAST: Faster Arbitrarily-Shaped Text Detector with Minimalist Kernel Representation”. Implementation based on the official Pytorch implementation: <czczup/FAST>`_.
>>> import torch >>> from doctr.models import textnet_base >>> model = textnet_base(pretrained=False) >>> input_tensor = torch.rand((1, 3, 512, 512), dtype=torch.float32) >>> out = model(input_tensor)
- vip_tiny(pretrained=False, **kwargs)[source]#
VIP-Tiny encoder architecture.Corresponds to SVIPTRv2-T variant in the paper (VIPTRv2 function in the official implementation: cxfyxl/VIPTR)
- vip_base(pretrained=False, **kwargs)[source]#
VIP-Base encoder architecture. Corresponds to SVIPTRv2-B variant in the paper (VIPTRv2B function in the official implementation: cxfyxl/VIPTR)
- crop_orientation_predictor(
- arch='mobilenet_v3_small_crop_orientation',
- pretrained=False,
- batch_size=128,
- **kwargs,
Crop orientation classification architecture.
>>> import numpy as np >>> from doctr.models import crop_orientation_predictor >>> model = crop_orientation_predictor(arch='mobilenet_v3_small_crop_orientation', pretrained=True) >>> input_crop = (255 * np.random.rand(256, 256, 3)).astype(np.uint8) >>> out = model([input_crop])
- Parameters:
arch (Any) – name of the architecture to use (e.g. ‘mobilenet_v3_small_crop_orientation’)
pretrained (bool) – If True, returns a model pre-trained on our recognition crops dataset
batch_size (int) – number of samples the model processes in parallel
**kwargs (Any) – keyword arguments to be passed to the OrientationPredictor
- Returns:
OrientationPredictor
- Return type:
OrientationPredictor
- page_orientation_predictor(
- arch='mobilenet_v3_small_page_orientation',
- pretrained=False,
- batch_size=4,
- **kwargs,
Page orientation classification architecture.
>>> import numpy as np >>> from doctr.models import page_orientation_predictor >>> model = page_orientation_predictor(arch='mobilenet_v3_small_page_orientation', pretrained=True) >>> input_page = (255 * np.random.rand(512, 512, 3)).astype(np.uint8) >>> out = model([input_page])
- Parameters:
arch (Any) – name of the architecture to use (e.g. ‘mobilenet_v3_small_page_orientation’)
pretrained (bool) – If True, returns a model pre-trained on our recognition crops dataset
batch_size (int) – number of samples the model processes in parallel
**kwargs (Any) – keyword arguments to be passed to the OrientationPredictor
- Returns:
OrientationPredictor
- Return type:
OrientationPredictor
doctr.models.detection#
- linknet_resnet18(pretrained=False, **kwargs)[source]#
LinkNet as described in “LinkNet: Exploiting Encoder Representations for Efficient Semantic Segmentation”.
>>> import torch >>> from doctr.models import linknet_resnet18 >>> model = linknet_resnet18(pretrained=True).eval() >>> input_tensor = torch.rand((1, 3, 1024, 1024), dtype=torch.float32) >>> out = model(input_tensor)
- linknet_resnet34(pretrained=False, **kwargs)[source]#
LinkNet as described in “LinkNet: Exploiting Encoder Representations for Efficient Semantic Segmentation”.
>>> import torch >>> from doctr.models import linknet_resnet34 >>> model = linknet_resnet34(pretrained=True).eval() >>> input_tensor = torch.rand((1, 3, 1024, 1024), dtype=torch.float32) >>> out = model(input_tensor)
- linknet_resnet50(pretrained=False, **kwargs)[source]#
LinkNet as described in “LinkNet: Exploiting Encoder Representations for Efficient Semantic Segmentation”.
>>> import torch >>> from doctr.models import linknet_resnet50 >>> model = linknet_resnet50(pretrained=True).eval() >>> input_tensor = torch.rand((1, 3, 1024, 1024), dtype=torch.float32) >>> out = model(input_tensor)
- db_resnet50(pretrained=False, **kwargs)[source]#
DBNet as described in “Real-time Scene Text Detection with Differentiable Binarization”, using a ResNet-50 backbone.
>>> import torch >>> from doctr.models import db_resnet50 >>> model = db_resnet50(pretrained=True) >>> input_tensor = torch.rand((1, 3, 1024, 1024), dtype=torch.float32) >>> out = model(input_tensor)
- db_mobilenet_v3_large(pretrained=False, **kwargs)[source]#
DBNet as described in “Real-time Scene Text Detection with Differentiable Binarization”, using a MobileNet V3 Large backbone.
>>> import torch >>> from doctr.models import db_mobilenet_v3_large >>> model = db_mobilenet_v3_large(pretrained=True) >>> input_tensor = torch.rand((1, 3, 1024, 1024), dtype=torch.float32) >>> out = model(input_tensor)
- fast_tiny(pretrained=False, **kwargs)[source]#
FAST as described in “FAST: Faster Arbitrarily-Shaped Text Detector with Minimalist Kernel Representation”, using a tiny TextNet backbone.
>>> import torch >>> from doctr.models import fast_tiny >>> model = fast_tiny(pretrained=True) >>> input_tensor = torch.rand((1, 3, 1024, 1024), dtype=torch.float32) >>> out = model(input_tensor)
- fast_small(pretrained=False, **kwargs)[source]#
FAST as described in “FAST: Faster Arbitrarily-Shaped Text Detector with Minimalist Kernel Representation”, using a small TextNet backbone.
>>> import torch >>> from doctr.models import fast_small >>> model = fast_small(pretrained=True) >>> input_tensor = torch.rand((1, 3, 1024, 1024), dtype=torch.float32) >>> out = model(input_tensor)
- fast_base(pretrained=False, **kwargs)[source]#
FAST as described in “FAST: Faster Arbitrarily-Shaped Text Detector with Minimalist Kernel Representation”, using a base TextNet backbone.
>>> import torch >>> from doctr.models import fast_base >>> model = fast_base(pretrained=True) >>> input_tensor = torch.rand((1, 3, 1024, 1024), dtype=torch.float32) >>> out = model(input_tensor)
- detection_predictor(
- arch='fast_base',
- pretrained=False,
- assume_straight_pages=True,
- preserve_aspect_ratio=True,
- symmetric_pad=True,
- batch_size=2,
- **kwargs,
Text detection architecture.
>>> import numpy as np >>> from doctr.models import detection_predictor >>> model = detection_predictor(arch='db_resnet50', pretrained=True) >>> input_page = (255 * np.random.rand(600, 800, 3)).astype(np.uint8) >>> out = model([input_page])
- Parameters:
arch (Any) – name of the architecture or model itself to use (e.g. ‘db_resnet50’)
pretrained (bool) – If True, returns a model pre-trained on our text detection dataset
assume_straight_pages (bool) – If True, fit straight boxes to the page
preserve_aspect_ratio (bool) – If True, pad the input document image to preserve the aspect ratio before running the detection model on it
symmetric_pad (bool) – if True, pad the image symmetrically instead of padding at the bottom-right
batch_size (int) – number of samples the model processes in parallel
**kwargs (Any) – optional keyword arguments passed to the architecture
- Returns:
Detection predictor
- Return type:
DetectionPredictor
doctr.models.recognition#
- crnn_vgg16_bn(pretrained=False, **kwargs)[source]#
CRNN with a VGG-16 backbone as described in “An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition”.
>>> import torch >>> from doctr.models import crnn_vgg16_bn >>> model = crnn_vgg16_bn(pretrained=True) >>> input_tensor = torch.rand(1, 3, 32, 128) >>> out = model(input_tensor)
- crnn_mobilenet_v3_small(pretrained=False, **kwargs)[source]#
CRNN with a MobileNet V3 Small backbone as described in “An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition”.
>>> import torch >>> from doctr.models import crnn_mobilenet_v3_small >>> model = crnn_mobilenet_v3_small(pretrained=True) >>> input_tensor = torch.rand(1, 3, 32, 128) >>> out = model(input_tensor)
- crnn_mobilenet_v3_large(pretrained=False, **kwargs)[source]#
CRNN with a MobileNet V3 Large backbone as described in “An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition”.
>>> import torch >>> from doctr.models import crnn_mobilenet_v3_large >>> model = crnn_mobilenet_v3_large(pretrained=True) >>> input_tensor = torch.rand(1, 3, 32, 128) >>> out = model(input_tensor)
- sar_resnet31(pretrained=False, **kwargs)[source]#
SAR with a resnet-31 feature extractor as described in “Show, Attend and Read:A Simple and Strong Baseline for Irregular Text Recognition”.
>>> import torch >>> from doctr.models import sar_resnet31 >>> model = sar_resnet31(pretrained=False) >>> input_tensor = torch.rand((1, 3, 32, 128)) >>> out = model(input_tensor)
- master(pretrained=False, **kwargs)[source]#
MASTER as described in paper: <https://arxiv.org/pdf/1910.02562.pdf>`_.
>>> import torch >>> from doctr.models import master >>> model = master(pretrained=False) >>> input_tensor = torch.rand((1, 3, 32, 128)) >>> out = model(input_tensor)
- vitstr_small(pretrained=False, **kwargs)[source]#
ViTSTR-Small as described in “Vision Transformer for Fast and Efficient Scene Text Recognition”.
>>> import torch >>> from doctr.models import vitstr_small >>> model = vitstr_small(pretrained=False) >>> input_tensor = torch.rand((1, 3, 32, 128)) >>> out = model(input_tensor)
- vitstr_base(pretrained=False, **kwargs)[source]#
ViTSTR-Base as described in “Vision Transformer for Fast and Efficient Scene Text Recognition”.
>>> import torch >>> from doctr.models import vitstr_base >>> model = vitstr_base(pretrained=False) >>> input_tensor = torch.rand((1, 3, 32, 128)) >>> out = model(input_tensor)
- parseq(pretrained=False, **kwargs)[source]#
PARSeq architecture from “Scene Text Recognition with Permuted Autoregressive Sequence Models”.
>>> import torch >>> from doctr.models import parseq >>> model = parseq(pretrained=False) >>> input_tensor = torch.rand((1, 3, 32, 128)) >>> out = model(input_tensor)
- viptr_tiny(pretrained=False, **kwargs)[source]#
VIPTR-Tiny as described in “A Vision Permutable Extractor for Fast and Efficient Scene Text Recognition”.
>>> import torch >>> from doctr.models import viptr_tiny >>> model = viptr_tiny(pretrained=False) >>> input_tensor = torch.rand((1, 3, 32, 128)) >>> out = model(input_tensor)
- recognition_predictor(
- arch='crnn_vgg16_bn',
- pretrained=False,
- symmetric_pad=False,
- batch_size=128,
- **kwargs,
Text recognition architecture.
- Example::
>>> import numpy as np >>> from doctr.models import recognition_predictor >>> model = recognition_predictor(pretrained=True) >>> input_page = (255 * np.random.rand(32, 128, 3)).astype(np.uint8) >>> out = model([input_page])
- Parameters:
arch (Any) – name of the architecture or model itself to use (e.g. ‘crnn_vgg16_bn’)
pretrained (bool) – If True, returns a model pre-trained on our text recognition dataset
symmetric_pad (bool) – if True, pad the image symmetrically instead of padding at the bottom-right
batch_size (int) – number of samples the model processes in parallel
**kwargs (Any) – optional parameters to be passed to the architecture
- Returns:
Recognition predictor
- Return type:
RecognitionPredictor
doctr.models.zoo#
- ocr_predictor(
- det_arch='fast_base',
- reco_arch='crnn_vgg16_bn',
- pretrained=False,
- pretrained_backbone=True,
- assume_straight_pages=True,
- preserve_aspect_ratio=True,
- symmetric_pad=True,
- export_as_straight_boxes=False,
- detect_orientation=False,
- straighten_pages=False,
- detect_language=False,
- **kwargs,
End-to-end OCR architecture using one model for localization, and another for text recognition.
>>> import numpy as np >>> from doctr.models import ocr_predictor >>> model = ocr_predictor('db_resnet50', 'crnn_vgg16_bn', pretrained=True) >>> input_page = (255 * np.random.rand(600, 800, 3)).astype(np.uint8) >>> out = model([input_page])
- Parameters:
det_arch (Any) – name of the detection architecture or the model itself to use (e.g. ‘db_resnet50’, ‘db_mobilenet_v3_large’)
reco_arch (Any) – name of the recognition architecture or the model itself to use (e.g. ‘crnn_vgg16_bn’, ‘sar_resnet31’)
pretrained (bool) – If True, returns a model pre-trained on our OCR dataset
pretrained_backbone (bool) – If True, returns a model with a pretrained backbone
assume_straight_pages (bool) – if True, speeds up the inference by assuming you only pass straight pages without rotated textual elements.
preserve_aspect_ratio (bool) – If True, pad the input document image to preserve the aspect ratio before running the detection model on it.
symmetric_pad (bool) – if True, pad the image symmetrically instead of padding at the bottom-right.
export_as_straight_boxes (bool) – when assume_straight_pages is set to False, export final predictions (potentially rotated) as straight bounding boxes.
detect_orientation (bool) – if True, the estimated general page orientation will be added to the predictions for each page. Doing so will slightly deteriorate the overall latency.
straighten_pages (bool) – if True, estimates the page general orientation based on the segmentation map median line orientation. Then, rotates page before passing it again to the deep learning detection module. Doing so will improve performances for documents with page-uniform rotations.
detect_language (bool) – if True, the language prediction will be added to the predictions for each page. Doing so will slightly deteriorate the overall latency.
kwargs (Any) – keyword args of OCRPredictor
- Returns:
OCR predictor
- Return type:
OCRPredictor
- kie_predictor(
- det_arch='fast_base',
- reco_arch='crnn_vgg16_bn',
- pretrained=False,
- pretrained_backbone=True,
- assume_straight_pages=True,
- preserve_aspect_ratio=True,
- symmetric_pad=True,
- export_as_straight_boxes=False,
- detect_orientation=False,
- straighten_pages=False,
- detect_language=False,
- **kwargs,
End-to-end KIE architecture using one model for localization, and another for text recognition.
>>> import numpy as np >>> from doctr.models import ocr_predictor >>> model = ocr_predictor('db_resnet50', 'crnn_vgg16_bn', pretrained=True) >>> input_page = (255 * np.random.rand(600, 800, 3)).astype(np.uint8) >>> out = model([input_page])
- Parameters:
det_arch (Any) – name of the detection architecture or the model itself to use (e.g. ‘db_resnet50’, ‘db_mobilenet_v3_large’)
reco_arch (Any) – name of the recognition architecture or the model itself to use (e.g. ‘crnn_vgg16_bn’, ‘sar_resnet31’)
pretrained (bool) – If True, returns a model pre-trained on our OCR dataset
pretrained_backbone (bool) – If True, returns a model with a pretrained backbone
assume_straight_pages (bool) – if True, speeds up the inference by assuming you only pass straight pages without rotated textual elements.
preserve_aspect_ratio (bool) – If True, pad the input document image to preserve the aspect ratio before running the detection model on it.
symmetric_pad (bool) – if True, pad the image symmetrically instead of padding at the bottom-right.
export_as_straight_boxes (bool) – when assume_straight_pages is set to False, export final predictions (potentially rotated) as straight bounding boxes.
detect_orientation (bool) – if True, the estimated general page orientation will be added to the predictions for each page. Doing so will slightly deteriorate the overall latency.
straighten_pages (bool) – if True, estimates the page general orientation based on the segmentation map median line orientation. Then, rotates page before passing it again to the deep learning detection module. Doing so will improve performances for documents with page-uniform rotations.
detect_language (bool) – if True, the language prediction will be added to the predictions for each page. Doing so will slightly deteriorate the overall latency.
kwargs (Any) – keyword args of OCRPredictor
- Returns:
KIE predictor
- Return type:
KIEPredictor
doctr.models.factory#
- from_hub(repo_id, **kwargs)[source]#
Instantiate & load a pretrained model from HF hub.
>>> from doctr.models import from_hub >>> model = from_hub("mindee/fasterrcnn_mobilenet_v3_large_fpn")
- push_to_hf_hub(model, model_name, task, **kwargs)[source]#
Save model and its configuration on HF hub
>>> from doctr.models import login_to_hub, push_to_hf_hub >>> from doctr.models.recognition import crnn_mobilenet_v3_small >>> login_to_hub() >>> model = crnn_mobilenet_v3_small(pretrained=True) >>> push_to_hf_hub(model, 'my-model', 'recognition', arch='crnn_mobilenet_v3_small')