ci(pdf-extract): add package verification gate

[stranske]

Source: Issue #2711

Closes #2711

Why

PDF text-EXTRACTION is independently reimplemented in four fleet repos with four divergent
result contracts and three OCR strategies (verified by reading source, 2026-06-28, and confirmed by a
fresh clone-grep across all branches — exactly four, none missed):

• Counter_Risk src/counter_risk/parsers/daily_holdings_pdf.py:81 (_extract_text ladder pdfplumber→pypdf→OCR).
• Pension-Data src/pension_data/parser/pdf_pipeline.py:455 (parse_pdf_to_funded_input) + src/pension_data/extract/orchestration/fallback.py:53 (run_fallback_chain) + db/models/provenance.py.
• Inv-Man-Intake src/inv_man_intake/extraction/providers/base.py:204 (ExtractionProvider Protocol; :218 MultiModalExtractionProvider) — current pdf_primary.py extractor is fixture-grade, not real.
• Manager-Database utils/extract.py:20 (_extract_pdf, pdfplumber→str).

Missing/duplicated behavior: items 1–4 of the shape (text-extraction ladder, OCR fallback, orchestration,
result/provenance contract) are rebuilt per repo; a reliability layer (arithmetic/business-rule validation +
cross-check + calibrated confidence) is absent in all four. This is latent fragility + duplicated effort,
not a current break. Grounding: Code/Audits/2026-06-28-fleet-pdf-extraction-survey.md and …-methodology.md.
A validated scaffold (this design) already exists at packages/stranske_pdf_extract/ with 27 passing
deterministic tests; this issue tracks landing it on main and finishing the optional-dep paths.

Scope

Land the single-source library stranske-pdf-extract as a pip-installable subdirectory package in
Workflows at packages/stranske_pdf_extract/ (distribution decision: package, NOT sync-manifest copy-sync —
see packages/stranske_pdf_extract/docs/DESIGN.md §2). The package owns: the generalized result + page-level
provenance contract (contract.py, generalizing Pension-Data + Inv-Man-Intake), the provider Protocols + OCR
seam + registry (provider.py), the fallback-ladder primitive (orchestration.py, lifted from Pension-Data),
the greenfield reliability layer (reliability.py), one REAL extractor behind the Protocol
(providers/docling_provider.py, Docling/MIT/local, optional [docling] extra), a pure-python baseline
(providers/text_baseline.py), and the golden-set eval harness (eval/harness.py). Tag pdf-extract-v0.1.0.

Non-Goals

• Do NOT add the package to .github/sync-manifest.yml — distribution is pip, not copy-sync. The only
  sync/doc touch is a note that the package exists and is pip-installed (so a future audit does not "fix" its absence).
• Do NOT run Docling or OCR in any stlite/Pyodide browser path; keep them optional extras never imported there.
• Do NOT migrate any consumer in this issue — migrations are separate, dependency-ordered tracking issues.
• Do NOT learn any consumer's domain schema; domain field-parsing stays in each consumer.
• Scaffold-only completion does NOT count: landing the tree with the named conformance gate collecting 0
  tests, or the deliberate-break below not demonstrated, is a failure of this issue.

Tasks

• Land packages/stranske_pdf_extract/ on main (the validated scaffold): pyproject.toml (extras
  baseline,docling,ocr,textract,schema,eval), src/stranske_pdf_extract/{contract,provider,orchestration,reliability}.py,
  providers/{docling_provider,text_baseline}.py, eval/harness.py, docs/DESIGN.md, README.md, and tests/.
• Wire the package's tests/ into Workflows CI (a non-default job, e.g. extend .github/workflows/selftest-ci.yml
  or add a packages-pdf-extract job) running PYTHONPATH=packages/stranske_pdf_extract/src python -m pytest packages/stranske_pdf_extract/tests.
• Tag the release pdf-extract-v0.1.0 and document the install URL
  git+https://github.com/stranske/Workflows@pdf-extract-v0.1.0#subdirectory=packages/stranske_pdf_extract in README.md.
• Complete the Docling real path (providers/docling_provider.py:_extract_real) behind the [docling] extra and
  add an opt-in test that runs only when docling_available() is true (skips cleanly otherwise).
• Perform the deliberate-break verification (see Acceptance Criteria), capture the FAIL output, then revert.

Acceptance Criteria

• Named test gate: packages/stranske_pdf_extract/tests/test_docling_provider.py::test_docling_provider_conforms_to_protocol
  passes in CI (asserts isinstance(DoclingProvider(), MultiModalExtractionProvider)), AND the full package suite
  collects a non-zero count and passes (confirmed-green locally: 27 passed).
• Deliberate-break gate: temporarily comment out the name = "docling" attribute in
  src/stranske_pdf_extract/providers/docling_provider.py (class DoclingProvider). With this change,
  test_docling_provider.py::test_docling_provider_conforms_to_protocol must FAIL with
  assert isinstance(...) == False. Revert and confirm it passes. (Demonstrated locally during scaffolding.)
• pip install "git+…#subdirectory=packages/stranske_pdf_extract" imports stranske_pdf_extract with the
  core (no extras) and import of providers does not require docling (only extract_modalities does).

Implementation Notes

• Confirmed-green local reproduction: cd packages/stranske_pdf_extract && PYTHONPATH=src python -m pytest -q → 27 passed.
• Core has zero runtime deps by design; native deps (Docling/Tesseract) are extras so confidentiality-sensitive
  consumers install only what they need.
• Contract merge rule and migration order are in packages/stranske_pdf_extract/docs/DESIGN.md.

Automated Status Summary

Scope

PDF text-EXTRACTION is independently reimplemented in four fleet repos with four divergent
result contracts and three OCR strategies (verified by reading source, 2026-06-28, and confirmed by a
fresh clone-grep across all branches — exactly four, none missed):

• Counter_Risk src/counter_risk/parsers/daily_holdings_pdf.py:81 (_extract_text ladder pdfplumber→pypdf→OCR).
• Pension-Data src/pension_data/parser/pdf_pipeline.py:455 (parse_pdf_to_funded_input) + src/pension_data/extract/orchestration/fallback.py:53 (run_fallback_chain) + db/models/provenance.py.
• Inv-Man-Intake src/inv_man_intake/extraction/providers/base.py:204 (ExtractionProvider Protocol; :218 MultiModalExtractionProvider) — current pdf_primary.py extractor is fixture-grade, not real.
• Manager-Database utils/extract.py:20 (_extract_pdf, pdfplumber→str).

Missing/duplicated behavior: items 1–4 of the shape (text-extraction ladder, OCR fallback, orchestration,
result/provenance contract) are rebuilt per repo; a reliability layer (arithmetic/business-rule validation +
cross-check + calibrated confidence) is absent in all four. This is latent fragility + duplicated effort,
not a current break. Grounding: Code/Audits/2026-06-28-fleet-pdf-extraction-survey.md and …-methodology.md.
A validated scaffold (this design) already exists at packages/stranske_pdf_extract/ with 27 passing
deterministic tests; this issue tracks landing it on main and finishing the optional-dep paths.

Tasks

• Land packages/stranske_pdf_extract/ on main (the validated scaffold): pyproject.toml (extras
  baseline,docling,ocr,textract,schema,eval), src/stranske_pdf_extract/{contract,provider,orchestration,reliability}.py,
  providers/{docling_provider,text_baseline}.py, eval/harness.py, docs/DESIGN.md, README.md, and tests/.
• Wire the package's tests/ into Workflows CI (a non-default job, e.g. extend .github/workflows/selftest-ci.yml
  or add a packages-pdf-extract job) running PYTHONPATH=packages/stranske_pdf_extract/src python -m pytest packages/stranske_pdf_extract/tests.
• Tag the release pdf-extract-v0.1.0 and document the install URL
  git+https://github.com/stranske/Workflows@pdf-extract-v0.1.0#subdirectory=packages/stranske_pdf_extract in README.md.
• Complete the Docling real path (providers/docling_provider.py:_extract_real) behind the [docling] extra and
  add an opt-in test that runs only when docling_available() is true (skips cleanly otherwise).
• Perform the deliberate-break verification (see Acceptance Criteria), capture the FAIL output, then revert.

Acceptance criteria

• Named test gate: packages/stranske_pdf_extract/tests/test_docling_provider.py::test_docling_provider_conforms_to_protocol
  passes in CI (asserts isinstance(DoclingProvider(), MultiModalExtractionProvider)), AND the full package suite
  collects a non-zero count and passes (confirmed-green locally: 27 passed).
• Deliberate-break gate: temporarily comment out the name = "docling" attribute in
  src/stranske_pdf_extract/providers/docling_provider.py (class DoclingProvider). With this change,
  test_docling_provider.py::test_docling_provider_conforms_to_protocol must FAIL with
  assert isinstance(...) == False. Revert and confirm it passes. (Demonstrated locally during scaffolding.)
• pip install "git+…#subdirectory=packages/stranske_pdf_extract" imports stranske_pdf_extract with the
  core (no extras) and import of providers does not require docling (only extract_modalities does).

[coderabbitai]

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 21 minutes

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: c5d50b2a-44e1-43ac-9ccd-3b40c63e1f56

📥 Commits

Reviewing files that changed from the base of the PR and between 46b1b3d and 497df98.

📒 Files selected for processing (6)

• .github/workflows/pr-00-gate.yml
• .github/workflows/selftest-ci.yml
• packages/stranske_pdf_extract/README.md
• packages/stranske_pdf_extract/docs/VERIFICATION.md
• packages/stranske_pdf_extract/pyproject.toml
• packages/stranske_pdf_extract/tests/conftest.py

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/issue-2711-pdf-extract-ci-release

---

_{Comment @coderabbitai help to get the list of available commands.}

[agents-workflows-bot]

🤖 Keepalive Loop Status

PR #2723 | Agent: Codex | Iteration 0/12

Current State

Metric	Value
Iteration progress	[----------] 0/12
Action	fix (agent-run-skipped)
Gate	failure
Tasks	0/8 complete
Timeout	45 min (default)
Timeout usage	11m elapsed (26%, 34m remaining)
Keepalive	✅ enabled
Autofix	❌ disabled

🔍 Failure Classification

| Error type | infrastructure |
| Error category | unknown |
| Suggested recovery | Capture logs and context; retry once and escalate if the issue persists. |

[github-actions]

Runner dispatch state for codex on PR #2723. Do not edit.

[agents-workflows-bot]

Keepalive Work Log (click to expand)

#	Time (UTC)	Agent	Action	Result	Files	Tasks	Progress	Commit	Gate
0	2026-06-30 11:29:25	Codex	wait (gate-cancelled-transient-transient)	skipped	—	0	0/8	—	cancelled
0	2026-06-30 11:30:54	Codex	run (agent-run-skipped)	skipped	—	0	0/8	—	cancelled
0	2026-06-30 11:33:17	Codex	wait (gate-not-success)	skipped	—	0	0/8	—	failure
0	2026-06-30 12:03:01	Codex	run (agent-run-skipped) retry	skipped	—	0	0/8	—	—
0	2026-06-30 12:04:13	Codex	run (agent-run-skipped)	skipped	—	0	0/8	—	success
0	2026-06-30 12:05:15	Codex	wait (gate-not-success)	skipped	—	0	0/8	—	failure
0	2026-06-30 12:30:04	Codex	run (agent-run-failed)	failure	3 file(s)	0	0/8	—	cancelled
0	2026-06-30 12:30:53	Codex	fix (agent-run-skipped)	skipped	—	0	0/8	—	failure
0	2026-06-30 12:31:30	Codex	wait (gate-pending-transient)	skipped	—	0	0/8	—	—
0	2026-06-30 12:33:20	Codex	fix (agent-run-skipped)	skipped	—	0	0/8	—	failure
0	2026-06-30 13:02:13	Codex	run (agent-run-skipped) retry	skipped	—	0	0/8	—	—
0	2026-06-30 13:04:11	Codex	fix (agent-run-skipped)	skipped	—	0	0/8	—	failure
0	2026-06-30 13:04:59	Codex	wait (gate-pending-transient)	skipped	—	0	0/8	—	—
0	2026-06-30 13:09:01	Codex	run (agent-run-failed)	failure	3 file(s)	0	0/8	—	cancelled
0	2026-06-30 13:09:50	Codex	run (agent-run-skipped) retry	skipped	—	0	0/8	—	—
0	2026-06-30 13:18:52	Codex	fix (agent-run-skipped) retry	skipped	—	0	0/8	—	failure
0	2026-06-30 13:19:37	Codex	run (agent-run-skipped) retry	skipped	—	0	0/8	—	—
0	2026-06-30 13:23:51	Codex	fix (agent-run-skipped) retry	skipped	—	0	0/8	—	failure
0	2026-06-30 13:28:24	Codex	run (agent-run-skipped) retry	skipped	—	0	0/8	—	—
0	2026-06-30 13:29:26	Codex	run (agent-run-skipped) retry	skipped	—	0	0/8	—	cancelled
0	2026-06-30 13:30:10	Codex	run (agent-run-skipped) retry	skipped	—	0	0/8	—	success
0	2026-06-30 14:08:57	Codex	run (agent-run-skipped) retry	skipped	—	0	0/8	—	cancelled
0	2026-06-30 14:23:12	Codex	run (agent-run-skipped) retry	skipped	—	0	0/8	—	—
0	2026-06-30 14:35:49	Codex	fix (agent-run-skipped)	skipped	—	0	0/8	—	failure
0	2026-06-30 14:36:54	Codex	run (agent-run-skipped) retry	skipped	—	0	0/8	—	—
0	2026-06-30 14:47:44	Codex	fix (agent-run-skipped)	skipped	—	0	0/8	—	failure

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a0ee3aa25c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[coderabbitai]

Caution

Failed to replace (edit) comment. This is likely due to insufficient permissions or the comment being deleted.

Error details

{}

[agents-workflows-bot]

🤖 Bot Comment Handler

• Agent: codex
• Bot comments to address: 1

The agent has been assigned to this PR to address the bot review comments.

Instructions for agent

1. Implement suggested fixes that improve the code
2. Skip suggestions that don't apply (note why in your response)

The bot comment handler workflow has prepared context in the artifacts.

[stranske]

Closer review fix pushed in f75bf848.

Addressed review feedback:

• Added packages-pdf-extract to .github/workflows/pr-00-gate.yml, so stranske-pdf-extract package tests is now part of the required Gate summary dependency chain instead of only Selftest CI.
• Kept the mirrored Selftest package job and added persist-credentials: false to the flagged checkout.
• Updated packages/stranske_pdf_extract/docs/VERIFICATION.md to state Gate enforces the package suite and Selftest mirrors it.

Local validation:

• PYTHONPYCACHEPREFIX=/tmp/imi-closer-pycache /opt/anaconda3/bin/python YAML parse for pr-00-gate.yml and selftest-ci.yml passed.
• PYTHONPYCACHEPREFIX=/tmp/imi-closer-pycache PYTHONPATH=src /opt/anaconda3/bin/python -m pytest tests -q in packages/stranske_pdf_extract -> 32 passed.
• git diff --check passed.

Post-push readback: head is f75bf848f7fcb5eb54476c285371df39f1eabea6; GitHub checks are pending on the fresh push; CodeRabbit reports pass. Per automation policy, leaving CI to settle for the next closer pass.

[stranske]

Follow-up security review fix pushed in 5272dcf0.

• Added explicit permissions: contents: read to the new Gate packages-pdf-extract job, addressing the broad-default-permissions zizmor finding.
• Revalidated workflow YAML parse for pr-00-gate.yml and selftest-ci.yml.
• Re-ran PYTHONPYCACHEPREFIX=/tmp/imi-closer-pycache PYTHONPATH=src /opt/anaconda3/bin/python -m pytest tests -q in packages/stranske_pdf_extract -> 32 passed.
• git diff --check passed.

[github-actions]

Provider Comparison Report

Provider Summary

Provider	Model	Verdict	Confidence	Summary
openai	gpt-5.4	CONCERNS	94%	This merged PR makes useful progress on the package verification gate: it extends selftest CI to run the stranske_pdf_extract package tests, adds test support via tests/conftest.py, and adds verifi...
anthropic	claude-sonnet-4-6	FAIL	88%	This PR adds CI wiring and a conftest.py for the pdf-extract package but critically omits the named test file test_docling_provider.py that is the primary acceptance criterion. The deliberate-bre...

📋 Full Provider Details (click to expand)

openai

• Model: gpt-5.4
• Verdict: CONCERNS
• Confidence: 94%
• Scores:
  • Correctness: 7.0/10
  • Completeness: 5.0/10
  • Quality: 8.0/10
  • Testing: 6.0/10
  • Risks: 7.0/10
• Summary: This merged PR makes useful progress on the package verification gate: it extends selftest CI to run the stranske_pdf_extract package tests, adds test support via tests/conftest.py, and adds verification documentation. The workflow wiring appears directionally correct and the code quality is reasonable. However, against the documented acceptance criteria, the implementation is incomplete: the diff does not show the core package landing work, does not clearly add a named targeted gate for the specific docling protocol test, and treats the deliberate-break requirement as documentation rather than executable verification. Because the changes only partially satisfy the requested acceptance scope, the appropriate verdict is CONCERNS rather than PASS.
• Concerns:
  • The PR only partially addresses the stated acceptance criteria for the pdf-extract verification gate. It adds workflow/test wiring and verification documentation, but does not land the core package files described in scope/tasks because those are not part of this diff.
  • The acceptance criterion requiring a named CI gate for packages/stranske_pdf_extract/tests/test_docling_provider.py::test_docling_provider_conforms_to_protocol is not clearly implemented in code. The workflow runs the full package test suite, but no targeted/named invocation for that specific test appears in the diff.
  • The deliberate-break verification is documented in packages/stranske_pdf_extract/docs/VERIFICATION.md rather than enforced or demonstrated by code/tests. Documentation alone does not prove the acceptance criterion was satisfied.
  • The requirement to tag release pdf-extract-v0.1.0 cannot be verified from code changes, and the README change shown is only +1 line; this is insufficient evidence that the release/install URL documentation requirement was fully addressed.
  • The docling real-path implementation behind the [docling] extra is not implemented in this PR diff. If it already existed before, this PR does not provide evidence for that acceptance item.
  • A non-pdf-related change to tools/llm_registry.py and its template/test was included. While likely harmless, it is unrelated to the acceptance criteria and increases review surface.

anthropic

• Model: claude-sonnet-4-6
• Verdict: FAIL
• Confidence: 88%
• Scores:
  • Correctness: 3.0/10
  • Completeness: 2.0/10
  • Quality: 5.0/10
  • Testing: 2.0/10
  • Risks: 6.0/10
• Summary: This PR adds CI wiring and a conftest.py for the pdf-extract package but critically omits the named test file test_docling_provider.py that is the primary acceptance criterion. The deliberate-break gate is unverifiable without the test and source files. The CI gate job in pr-00-gate.yml has a failing test-quality job. The PR is substantially incomplete relative to its stated acceptance criteria.
• Concerns:
  • The primary acceptance criterion requires test_docling_provider.py::test_docling_provider_conforms_to_protocol to exist and pass — this file is entirely absent from the diff
  • The deliberate-break gate cannot be verified because neither test_docling_provider.py nor providers/docling_provider.py appear in the changes
  • Only conftest.py is added to the test suite; no actual test file for the named gate is present
  • CI workflow pr-00-gate.yml shows a test-quality job failure, which is a disqualifying result per evaluation instructions
  • Unrelated changes (llm_registry.py, test_langchain_client.py) are bundled into this PR without clear justification
  • No evidence of the pdf-extract-v0.1.0 tag or updated README install URL as required by the tasks
  • The VERIFICATION.md documents the verification but cannot substitute for the actual test implementation

Agreement

• Risks: scores within 1 point (avg 6.5/10, range 6.0-7.0)

Disagreement

Dimension	openai	anthropic
Verdict	CONCERNS	FAIL
Correctness	7.0/10	3.0/10
Completeness	5.0/10	2.0/10
Quality	8.0/10	5.0/10
Testing	6.0/10	2.0/10

Unique Insights

• openai: The PR only partially addresses the stated acceptance criteria for the pdf-extract verification gate. It adds workflow/test wiring and verification documentation, but does not land the core package files described in scope/tasks because those are not part of this diff.; The acceptance criterion requiring a named CI gate for packages/stranske_pdf_extract/tests/test_docling_provider.py::test_docling_provider_conforms_to_protocol is not clearly implemented in code. The workflow runs the full package test suite, but no targeted/named invocation for that specific test appears in the diff.; The deliberate-break verification is documented in packages/stranske_pdf_extract/docs/VERIFICATION.md rather than enforced or demonstrated by code/tests. Documentation alone does not prove the acceptance criterion was satisfied.; The requirement to tag release pdf-extract-v0.1.0 cannot be verified from code changes, and the README change shown is only +1 line; this is insufficient evidence that the release/install URL documentation requirement was fully addressed.; The docling real-path implementation behind the [docling] extra is not implemented in this PR diff. If it already existed before, this PR does not provide evidence for that acceptance item.; A non-pdf-related change to tools/llm_registry.py and its template/test was included. While likely harmless, it is unrelated to the acceptance criteria and increases review surface.
• anthropic: The primary acceptance criterion requires test_docling_provider.py::test_docling_provider_conforms_to_protocol to exist and pass — this file is entirely absent from the diff; The deliberate-break gate cannot be verified because neither test_docling_provider.py nor providers/docling_provider.py appear in the changes; Only conftest.py is added to the test suite; no actual test file for the named gate is present; CI workflow pr-00-gate.yml shows a test-quality job failure, which is a disqualifying result per evaluation instructions; Unrelated changes (llm_registry.py, test_langchain_client.py) are bundled into this PR without clear justification; No evidence of the pdf-extract-v0.1.0 tag or updated README install URL as required by the tasks; The VERIFICATION.md documents the verification but cannot substitute for the actual test implementation

🔍 LangSmith Traces

• openai
• anthropic