Add support for olmOCR integration

[eliasjudin]

Requested feature

Integrate olmOCR into Docling to enhance OCR capabilities, particularly for complex PDFs containing tables, equations, and handwritten notes.

[neuralmaticv]

Has anyone already experimented with integrating olmOCR into Docling?
I'm considering adding it as an optional backend using the OCR factory.

@dosu, could you share a bit more information or guidance on implementing a custom OCR plugin (beyond the general plugin docs)?

[dosubot]

Hi @neuralmaticv! I'm Dosu and I’m helping the docling team.

There’s active interest and recent work on integrating olmOCR into Docling, but there’s no evidence of a fully completed or released plugin yet. The plugin system is designed for this use case and supports third-party OCR backends via pluggy/setuptools entry points. To implement a custom OCR plugin for olmOCR, you’ll want to:

1. Create a model class inheriting from BaseOcrModel and implement __init__, __call__, and a classmethod get_options_type that returns your options class.
2. Define an options class inheriting from OcrOptions, with a kind class variable matching your plugin entry point name.
3. In your plugin module, implement a function ocr_engines() that returns a dict with the key "ocr_engines" and a list of your OCR engine classes as the value.
4. Package your plugin as a separate Python project, and in pyproject.toml, register your entry point under [project.entry-points."docling"].
5. Install your plugin and enable external plugins in Docling by setting pipeline_options.allow_external_plugins = True or using the CLI flag.

You can find a detailed example and template in the Docling issues, and the default plugin code is a good reference for structure. The plugin docs also walk through the registration and loading process.

A key caveat: olmOCR’s output is JSON and may not include bounding boxes, which Docling expects for some OCR tasks. You’ll need to handle this in your plugin’s __call__ method—parse the JSON, extract the recognized text, and (if possible) map any available layout info to Docling’s TextItem objects. See this PR for an example of decoding olmOCR’s JSON output and integrating it into Docling’s pipeline.

If you want a minimal template or have questions about specific olmOCR quirks, let me know!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}