surya-rs

Rust implementation of surya, a multilingual document OCR toolkit. The implementation is based on a modified version of Segformer, OpenCV, and donut transformer.

Please refer to the original project for more details on licensing of the weights.

Roadmap

This project is still in development, feel free to star and check back.

• image input pre-processing
• detection - segformer
• detection - weights loading
• detection - heatmap and affinity map
• detection - bboxes
• detection - image splitting and stitching
• recognition - swin encoder
• recognition - MoE MBart
• recognition - donut transformer loading
• benchmark
• quantifications

How to build and install

Setup rust toolchain if you haven't yet:

# visit https://rustup.rs/ for more detailed information
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Install llvm and opencv (example on Mac):

brew install llvm opencv

Build and install the binary:

# run this first on Mac if you have a M1 chip
export DYLD_FALLBACK_LIBRARY_PATH="$(xcode-select --print-path)/usr/lib/"
# run this first on other Mac
export DYLD_FALLBACK_LIBRARY_PATH="$(xcode-select --print-path)/Toolchains/XcodeDefault.xctoolchain/"
# optionally you can include features like accelerate, metal, mkl, etc.
cargo install --path . --features=cli

The binary when built does not include the weights file itself, and will instead download via the HuggingFace Hub API. Once downloaded, the weights file will be cached in the HuggingFace cache directory.

Check -h for help:

Surya is a multilingual document OCR toolkit, original implementation in Python and PyTorch

Usage: surya [OPTIONS] <IMAGE>

Arguments:
  <IMAGE>  path to image

Options:
      --detection-batch-size <DETECTION_BATCH_SIZE>
          detection batch size, if not supplied defaults to 2 on CPU and 16 on GPU
      --detection-model-repo <DETECTION_MODEL_REPO>
          detection model's hugging face repo [default: vikp/surya_det]
      --weights-file-name <WEIGHTS_FILE_NAME>
          detection model's weights file name [default: model.safetensors]
      --config-file-name <CONFIG_FILE_NAME>
          detection model's config file name [default: config.json]
      --non-max-suppression-threshold <NON_MAX_SUPPRESSION_THRESHOLD>
          a value between 0.0 and 1.0 to filter low density part of heatmap [default: 0.35]
      --extract-text-threshold <EXTRACT_TEXT_THRESHOLD>
          a value between 0.0 and 1.0 to filter out bbox with low heatmap density [default: 0.6]
      --bbox-area-threshold <BBOX_AREA_THRESHOLD>
          a pixel threshold to filter out small area bbox [default: 10]
      --recognition-batch-size <RECOGNITION_BATCH_SIZE>
          recognition batch size, if not supplied defaults to 8 on CPU and 256 on GPU
      --recognition-model-repo <RECOGNITION_MODEL_REPO>
          recognition model's hugging face repo [default: vikp/surya_rec]
      --output-dir <OUTPUT_DIR>
          output directory, under which the input image will be generating a subdirectory [default: ./surya_output]
      --polygons
          whether to output polygons json file
      --image
          whether to generate bbox image
      --heatmap
          whether to generate heatmap
      --affinity-map
          whether to generate affinity map
      --device <DEVICE_TYPE>
          device type, if not specified will try to use GPU or Metal [possible values: cpu, gpu, metal]
      --verbose
          whether to enable verbose mode
  -h, --help
          Print help
  -V, --version
          Print version

You can also use this to control logging level:

export SURYA_LOG=warn # or debug, warn, etc.

Library

This lib is also published as a trait for other rust projects to use.