Repository navigation
Replies: 3 comments
|
@Caldalis the shape is right: the engine would land in content-core and Open Notebook would only select it, which is how extraction is split today. Before we go further, though, I need evidence that it's worth a new engine. 1. Show that it beats what we have. The case rests on "Docling's local OCR struggles with difficult scans, tables, formulas and CJK", but there's no comparison in the thread. Could you run both engines on the same small set (say 5 to 10 documents: a poor scan, a photographed page, a table-heavy PDF, a page with formulas, a CJK document) and post:
Public benchmarks are welcome as context, but a side-by-side on documents like the ones our users ingest is what would decide this. 2. Dedicated or generic. Open Notebook keeps vendor-specific behaviour out of the core unless there's a deliberate reason (PDR-002). For Docling picture description we went generic: any OpenAI-compatible vision endpoint (#1478). If the gain holds, the next question is whether it comes from the model (then a generic "OCR through any vision endpoint" engine would carry it) or from the MiniCPM-OCR service's own document API (async jobs, reading order, HTML tables), which would justify a dedicated adapter. On your three questions, for when this moves forward: This Discussion is the home for the OCR slice of #331. Status: exploring. Waiting on the side-by-side comparison. |
|
Thanks — I ran the same documents through Docling, the MiniCPM-OCR service, and the MiniCPM model alone (one full-page image through the vLLM endpoint). Ten one-page documents covering the categories you listed. Full outputs and numbers: https://gist.github.com/Caldalis/a7adcf1ae56960753825ba4141f288e5 Hardware: shared 8× RTX 4090, 96-core Xeon. Docling ran with CUDA available; MiniCPM used its GPU service plus CPU layout. One run after warm-up, so the timings are indicative. Where MiniCPM is clearly better:
Where Docling is still better:
Model vs service: the model alone is faster, but it loses table structure and reading order and makes more OCR errors. The gain comes from the service pipeline — orientation/deskew, layout and reading order, per-region prompts, HTML tables, formula assembly. That is why I think a dedicated adapter makes sense rather than a generic vision endpoint. One honest failure: on a single-line 90°-rotated scan the service returned success with zero text (layout found no regions), while Docling and the model-alone path both read it. content-core treats an empty result as an error, so the user sees a failure rather than an empty source. Scope: I'll keep the first PR self-hosted only, environment-only configuration, with the Caveats: one page per document, one run after warm-up; the CJK and degraded-scan pages are synthetic, the rest are Docling's public test documents. |
|
@Caldalis thank you, this is the comparison I asked for, and it's an honest one: you listed your own engine's failures next to Docling's. What I take from the gist:
One thing the gist doesn't state: how Docling was configured. Was OCR enabled, and with which OCR engine? The receipt result (placeholders only) reads like OCR didn't run on the image, and that changes the size of the gap. Where this goes next. The engine itself belongs in content-core, not in Open Notebook, so this conversation is moving there. Rather than add one more built-in engine, I've opened a broader thread on letting engines ship as separate Python packages that content-core discovers, the way LangChain handles partner packages: lfnovo/content-core#116. MiniCPM-OCR is named there as the first concrete candidate, and your adapter is a good case to design the contract against (it needs a service URL, it has a clear "not for Please continue there instead of here: what your adapter needs from a plugin contract is exactly the input that thread is asking for. Open Notebook picks an engine up once content-core makes it available; nothing needs to be decided on this side before then. Status: combine → lfnovo/content-core#116 |
Uh oh!
There was an error while loading. Please reload this page.
Problem
Scanned PDFs, photographed pages, and image-only documents still extract poorly. Docling's local OCR is the only OCR path today; it is CPU-bound and struggles with difficult scans, complex tables, formulas, and CJK documents. Users who already run a MiniCPM-OCR service (or use the ModelBest API) have no way to plug it into Open Notebook.
Proposal
Add MiniCPM-OCR as an optional document engine, following the same split as other extraction features:
document_engine="minicpm_ocr"processor. It sends the file to a MiniCPM-OCR service over HTTP and returns Markdown (reading order, HTML tables, LaTeX formulas). Two service flavors are supported:openbmbofficial/minicpm-ocrimage's document API;CCORE_MINICPM_OCR_URL(the same env-first pattern as the Docling vision endpoint in feat(extraction): configurable vision model for Docling picture description (OpenAI-compatible endpoint) #1478), documents the variables, and leavesautounchanged.Configuration:
CCORE_MINICPM_OCR_URL(required), plus optional_SERVICE(self_hosted/modelbest),_API_KEY,_MODEL,_UPRIGHT,_TIMEOUT, and_POLL_INTERVAL.Why this shape
autonever picks it.ConfigurationError, unreachable service →NetworkError, failed or empty job →ExternalServiceError.Scope
Alternatives considered
Questions
minicpm_ocrthe right engine name, or would you preferminicpm?All reactions