PyMuPDF with tesseract OCR as External Content Extraction Engine

[rgaricano]

Example of External Content Extraction Engine

PyMuPDF with tesseract OCR as External Content Extraction Engine

Default Content Extraction Engine use pypdf for extract text from pdf files.
The inconvenient is that it doesn't can proccess images or do OCR over its.

Pymudf does it, throught tesseract (that also have GPU support), but there are Open-WebUI licenses incompatibilities for use it as integrated lib.

For that reason I implemented it as a External Content Extraction Engine.
The inconvenient is that when it's that the external Content Extraction Engine is not per file type, it have to manage all other file types too.
For solve it I added all other file types proccessing that are natively by default in Open-WebUI (or so I think, maybe some have escaped!) & also I improved it with:

• Pdf blocks detection and OCR over blocks that haven't text layer (also configurable minimal text in layer for trigger OCR).
• OCR for uploaded images (jpg, jpeg, png, tiff, bmp, webp, gif).
• configurable language support.

Basically is a FastAPI HTTP service that implements the same API as expected by ExternalDocumentLoader.

Installation for linux (ubuntu):

Install tesseract plus (additional languages pack if necessary):

• tesseract: sudo apt install tesseract-ocr libtesseract-dev
• language pack, e.g. for spanish: sudo apt install tesseract-ocr-spa (https://tesseract-ocr.github.io/tessdoc/Data-Files.html)
• set enviroment var for use it (for ubuntu: export TESSDATA_PREFIX="/usr/share/tesseract-ocr/5/tessdata" )
• set enviroment var for use GPU: export CUDA_VISIBLE_DEVICES="0"
• (for global set, edit /etc/enviroment and add its, then reload enviroment set -a; source /etc/environment; set +a;)

Install the pyMuPDF service:

• Create a directory and place in it the attached files requiremenst.txt & pymupdf_service.py
• Enable python virtual enviroment. (python3 -m venv venv & source venv/bin/activate)
• Install requirements.txt (pip install -r requirements.txt)
• Run service (python pymupdf_service.py)

Configurable parameters:

--host ("Host to bind to (default: 0.0.0.0)")
--port ("Port to bind to (default: 8000)")
--min-text-length ("Minimum text length for PDF text layer detection (default: 50)")
--ocr-language ("OCR language code for PDFs (default: eng)")

Examples (for run the content extraction engine server):

# Using command line arguments  
python pymupdf_service.py --ocr-language "spa" --port 8090  
  
# Multiple languages  
python pymupdf_service.py --ocr-language "eng+spa+fra" --port 8090  
  
# Using environment variables  
OCR_LANGUAGE="deu" PORT=8090 python pymupdf_service.py  
  
# Default (English)  
python pymupdf_service.py

pymupdf_service.py
requirements.txt

Tested with Open-WebUI v0.6.30

---

I don't use docker, but I left a dockfile and a yml with this service and with tesseract, if someone want to try or improve it. I didn't test its!!! (and it include all possible env vars, most of its unnecessaries)

Its need to be adapted to one's own needs!! (as language pack, ...)

docker-compose.gpu.yml
dockerfile.txt

[rgaricano]

For better results with OCR replace lines 219 & 577 of pymupdf_service.py:
text = page.get_text(textpage=tp)
by
text = page.get_text("text", textpage=tp, flags=pymupdf.TEXT_DEHYPHENATE | pymupdf.TEXT_PRESERVE_WHITESPACE | pymupdf.TEXT_MEDIABOX_CLIP)

This prevent the line fragmentation.

(https://github.com/pymupdf/PyMuPDF/blob/78b39c13/docs/vars.rst#text-extraction-flags )

[rgaricano]

when you launch the server (python pymupdf_service.py) it run in a host:port (that you can define), that is the url to put as external extraction engine url, if there isn't any specific by default it run in http://localhost:8000
the api key is not neccessary, is not used in this sample, you can put any.

[ghost]

@rgaricano thank you! It worked.

Would there be plan to natively support OCR via Tesseract/PyMuPDF/Stirling PDF? I have tried little switch "PDF Extract Images (OCR)" but it doesn't work on scanned pdfs.

[rgaricano]

I don't think so,
I made a PR for use pymupdf instead pypdf but was a license incompatibility. With tesseract there isn't that problem, it could be used for OCR images (or pdf converted to images) but it's necessary a native install of tesseract...that it's another inconvenient.
Always we can provide a more elaborate external engine, as this....and is the same situation of all other "engines" as tabily or docling, even though they may seem "integrated", they are external services that require their own separate installation.

My idea was develop the external engine with a wider configuration options, e.g. can choose libs for each extension as pandas, openpyxl, python-docx, openpyxl, python-pptx, odfpy, ebooklib, lxml, xlrd,...

[ghost]

Make sense.

Would appreciate to have Stirling PDF as an external engine. Perhaps an ability to have fallback external engine would be cherry on the top.

[rgaricano]

Not sure if it's apropiate add Stirling PDF as engine, it "overpass" that funcionality, but as it use libreoffice unoserver for conversions maybe is a good idea look at it (unoserver) as an option for document extraction.

[Arien02]

Just a quick "thank you" (¡gracias!) for this work, I've found it very usefull, but at least with OWUI v0.8.5 it doesn't work, soy i've modded it introducing this code:

try:
        # Tu línea original que llama al procesador
        documents = processor.process_document(temp_file_path, filename, content_type)

        # --- AÑADE ESTO AQUÍ (LIMPIEZA DE SEGURIDAD FINAL) ---
        if isinstance(documents, list):
            for doc in documents:
                if "metadata" in doc:
                    doc["metadata"] = {k: v for k, v in doc["metadata"].items() if v is not None}
        elif isinstance(documents, dict) and "metadata" in documents:
            documents["metadata"] = {k: v for k, v in documents["metadata"].items() if v is not None}
        # ----------------------------------------------------

        # Tu return original
        if len(documents) == 1:
            return JSONResponse(content=documents[0])
        else:
            return JSONResponse(content=documents)

So, if anyone find this usefull like me, here's the modded service.py
pymupdf_service.py

[sanchitbhavsar]

@rgaricano I am trying this approach and running the app locally and via docker both i can curl the endpoint to process the document and it works BUT i cannot make it work with the openwebui. I did the general settings for Content Extraction Engine as External and with the host http://localhost:8090 for local and http://host.internal.docker:8000 both of them did not call this pymupdf service for processing the document. What else am i missing here? I use v0.8.12

Also tried the updated code from @Arien02 but no result.