fix(parser): bound OCR worker init so offline scanned PDFs degrade, never hang (#298)

[thewrz]

Why

The PDF OCR fallback (#246/#290) initializes the Tesseract worker via
Tesseract.createWorker('eng', 1, …). When the model cache is empty and
OCR_LANG_PATH is unset, tesseract.js fetches eng.traineddata from a CDN on
first use. Offline, two failure modes exist:

1. Fail-fast (already handled, feat(parser): PDF OCR fallback + font-encoding recovery (#246) #290): the fetch rejects (TypeError: fetch failed) → clean pdf-ocr-unusable warning.
2. Indefinite stall (was NOT handled): if the network accepts the connection
   but never responds, createWorker neither resolves nor rejects, and the first
   scanned-PDF parse job hangs forever — wedging a Piscina worker thread.

There was no bounded init timeout on the production OCR path.

What

• Bounded worker init (the spine). initManagedRecognizer races worker
  creation against a configurable timer. On timeout it rejects with a typed
  ParserError, which the existing applyOcrIfNeeded catch degrades to a
  pdf-ocr-unusable warning — so a stall now completes within a bounded time
  instead of hanging. A late-resolving worker is terminated (terminateLater) so
  no Tesseract process leaks.
• New env knob OCR_INIT_TIMEOUT_MS (Zod-validated, positive int, default
  30000), threaded parse-worker → ParseOptions → PdfOcrOptions.initTimeoutMs.
  Documented in .env.example.
• feat(parser): PDF OCR fallback + font-encoding recovery (#246) #290 fail-fast preserved. Error wrapping is centralized at the init
  boundary, so a rejecting init still surfaces a ParserError → pdf-ocr-unusable.
• DI seam PdfOcrOptions.createWorker (mirrors the existing
  renderPageAsImage / recognize seams) makes the stall unit-testable by
  simulating a never-resolving init — no network, no real traineddata.

Design decisions

The issue floated three options (bounded init timeout, local traineddata,
pre-flight check). ADR-039 records the call:

• Bounded init timeout = the load-bearing fix. Lowest-risk, and it subsumes
  every "model not readily available" case (absent, unreadable, or hung) with
  one mechanism. It is the only thing added to the hot path.
• Local traineddata = the documented production requirement. Production OCR
  must not depend on a CDN: vendor eng.traineddata (Apache-2.0) and point
  OCR_LANG_PATH at it. The timeout is the safety net for when that is missed.
  The ~15 MB model is an ops artifact and is not committed to the repo.
• Pre-flight check = rejected. It would duplicate tesseract.js's own
  resolution logic, risk drifting from it, and still not cover a path that exists
  but whose fetch stalls — which the timeout already covers.

See docs/adr/039-offline-ocr-provisioning.md for the full Context/Decision/
Consequences, building on ADR-034.

Testing

• pnpm lint clean (eslint + tsc + prettier)
• pnpm test — 1185 unit tests pass (with DATABASE_URL/NODE_ENV set, as CI does)
• New regression test: ocr: worker init stall degrades to pdf-ocr-unusable within timeout, never hangs (parsePdf boundary) + focused recognizePdfPages tests for ParserError-on-stall, feat(parser): PDF OCR fallback + font-encoding recovery (#246) #290 fail-fast preservation, and no-leak late-worker termination — all offline, no real traineddata
• CI green
• Real-OCR e2e (SPECR_OCR_E2E=1) unaffected — the bounded-timeout tests do not require network or real traineddata

🤖 Co-authored by Claude Opus 4.8 (1M context). Closes #298.

Summary by CodeRabbit

• New Features
  • Added OCR settings for worker init timeout and strict local-traineddata mode.
  • Improved offline OCR support by allowing local model checks before starting OCR processing.
• Bug Fixes
  • Prevents OCR from hanging when worker initialization stalls.
  • Avoids worker leaks and fails gracefully when OCR worker startup or model availability checks fail.
  • Updates OCR handling so missing local training data no longer triggers CDN-dependent behavior in restricted environments.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: e8d2eb62-44f6-43e6-81c6-7e164e434913

📥 Commits

Reviewing files that changed from the base of the PR and between 1536323 and 5dcf1f0.

📒 Files selected for processing (8)

• .env.example
• docs/adr/039-offline-ocr-provisioning.md
• src/lib/env.ts
• src/lib/parse-worker.ts
• src/parser/index.ts
• src/parser/pdf/index.test.ts
• src/parser/pdf/ocr.test.ts
• src/parser/pdf/ocr.ts

---

📝 Walkthrough

Walkthrough

Adds two complementary offline OCR safety mechanisms: a configurable worker-initialization timeout (OCR_INIT_TIMEOUT_MS, default 30s) that races Tesseract worker creation and degrades to pdf-ocr-unusable on stall, and an opt-in strict pre-flight (OCR_REQUIRE_LOCAL_TRAINEDDATA) that refuses to spawn a worker when local traineddata is absent. Config propagation, tests, .env.example, and ADR-039 are included.

Changes

Offline OCR provisioning

Layer / File(s)	Summary
OCR init timeout, pre-flight, and traineddata probe src/parser/pdf/ocr.ts	Expands PdfOcrOptions with initTimeoutMs, createWorker, requireLocalTraineddata, and hasLocalTraineddata DI fields. Exports ManagedRecognizer, ManagedRecognizerFactory, and LocalTraineddataCheck. Adds DEFAULT_OCR_INIT_TIMEOUT_MS = 30_000, refactors worker creation into a timeout-raced initManagedRecognizer (terminates late-resolving workers to prevent leaks), and adds hasLocalTraineddata probe plus assertLocalTraineddataIfRequired pre-flight invoked from withRecognizer.
Config schema, ParseOptions, and worker wiring src/lib/env.ts, src/parser/index.ts, src/lib/parse-worker.ts	Extends Zod env schema with OCR_INIT_TIMEOUT_MS and OCR_REQUIRE_LOCAL_TRAINEDDATA. Adds matching optional fields to ParseOptions, updates ocrOptionsFromParseOptions with a hasAnyOcrOption helper that includes the new fields, and propagates them through parseOptionsFromConfig.
Tests: timeout, pre-flight, and traineddata probe src/parser/pdf/ocr.test.ts, src/parser/pdf/index.test.ts	ocr.test.ts covers timeout-based ParserError, worker termination on late resolution, reject fail-fast, strict-offline pre-flight that blocks createWorker, and hasLocalTraineddata detection of .traineddata/.traineddata.gz. index.test.ts adds three parsePdf integration tests for the same degradation paths.
ADR-039 and .env.example docs/adr/039-offline-ocr-provisioning.md, .env.example	ADR-039 documents the offline provisioning decision, traineddata probing rules, consequences, rejected alternatives, and a revision noting the worker-leak gap not addressed by timeout alone. .env.example adds the two new settings with expanded CDN-avoidance guidance.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• wrzonance/SpecR#290: Introduced the OCR fallback pipeline in src/parser/pdf/ocr.ts and the pdf-ocr-unusable degradation path that this PR extends with timeout and pre-flight logic.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 21.43% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: bounded OCR init with offline scan degradation instead of hangs.
Linked Issues check	✅ Passed	The changes satisfy #298 by adding bounded OCR init, offline local-traineddata handling, and ADR-039 documentation.
Out of Scope Changes check	✅ Passed	The added docs, config, types, and tests all support the OCR timeout/offline provisioning scope.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/issue-298

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[thewrz]

@coderabbitai full review

[coderabbitai]

✅ Action performed

Full review finished.