MIB Doc Challenge — offline adjudication pipeline

An offline, CPU-only pipeline for the MIB Doc Challenge. It reads a directory of PDF case packets and writes one JSONL prediction per packet: nine extracted fields, an APPROVED / DENIED / NEEDS_REVIEW decision, and a calibrated confidence.

No LLM, VLM, API, or network call is used at runtime.

Measured result

Scored on all 1,000 public training packets with the organizer's scripts/evaluate.py:

Section	Score
Field extraction	42.16 / 50
Classification	65.37 / 80
Confidence calibration	16.19 / 20
Total	123.72 / 150
Catastrophic false approvals	9

The run produced 1,000 valid records with no missing, extra, duplicate, or schema-invalid cases. It completed in 896.71 seconds on the development host (0.897 seconds/PDF, four workers).

Build and run offline

docker build --platform linux/amd64 -t mib-submission .

mkdir -p /tmp/mib-output
docker run --rm \
  --network none \
  --cpus 4 \
  --memory 8g \
  --pids-limit 512 \
  --read-only \
  --tmpfs /tmp:rw,nosuid,nodev,size=2g \
  --mount type=bind,src="/path/to/pdfs",dst=/input,readonly \
  --mount type=bind,src="/tmp/mib-output",dst=/output \
  mib-submission /input /output/predictions.jsonl

The image accepts exactly two arguments:

<input_pdf_dir> <output_predictions_path>

--platform linux/amd64 matters when building on an Arm host because the grader runs x86-64. The container writes temporary files only under /tmp and final output only at the requested path, so it runs with a read-only root filesystem.

For local development:

python3 -m venv .venv
./.venv/bin/pip install -r requirements.txt
./.venv/bin/python -m mib.cli <input_pdf_dir> <output_predictions_path>

Tesseract and Poppler must be on PATH for the OCR fallback.

Design

The corpus contains adversarial and damaged documents, so the pipeline separates evidence recovery from decision-making:

• mib/ingest.py enforces the trust boundary. White-on-white, tiny, off-crop, and fake-answer-key text is quarantined before extraction. Multi-line visible answer keys are removed as bounded blocks.
• Pages are typed from their printed titles. Conflicting values resolve using the field manual's authority order—adjudicator note, intake form, biometric slip, sponsor attestation, registry extract—before confidence or page order.
• mib/ocr.py runs three unioned Tesseract passes only on pages whose visible text layer is insufficient.
• mib/roi_ocr.py gives decisive template regions a second, high-resolution reading. A four-reading biometric ensemble selects one strongest supported risk reading; it never unions alternatives. Each crop is field-filtered before it can reach extraction, so an OCR variant of a neighboring field cannot override correct native text. Intake ROI may fill a packet-level blank species, visa, purpose, or arrival field, but cannot compete with any value already recovered before ROI. On damaged fee receipts, a separate multi-reading line ensemble may fill a still-blank packet fee only when at least two readings support one canonical status and no conflicting status is tied; existing ordinary, legacy-ROI, or manually corrected fee evidence always wins.
• mib/extract.py and mib/normalize.py use label geometry, bounded fuzzy repair, field vocabularies, and explicit handling for manual corrections and sponsor-attestation prose. If exact finding parsing fails on a page already typed as a rank-1 adjudicator note, a conservative label-anchored fuzzy pass may recover one unique decision; conflicts abstain. A unique visible date on a form-like page may improve the emitted extraction, but remains zero-confidence and cannot become policy evidence when its arrival label was not recovered.
• mib/policy.py evaluates the declarative cascade in rules/policy.yaml. mib/decide.py chooses the action with the highest expected challenge score, accounting for the asymmetric false-approval penalty.
• Runtime posteriors in rules/posteriors.json are fitted on the pipeline's own extracted fields, so confidence reflects observed extraction failures rather than an assumption of perfect transcription.

The measurements behind these choices, including rejected approaches, are in docs/RECON.md.

Attribution

Public challenge participants whose submissions and research informed the comparative experiments are credited, with source links, in ATTRIBUTION.md.

Reproduce checks

From this repository, with the challenge repository at ../mib-doc-challenge:

./.venv/bin/python -m unittest discover -s tests
make policy
make score
make validate

Expensive PDF/OCR evidence can be cached once and replayed while developing extractors, policy, or posteriors:

./.venv/bin/python scripts/cache_evidence.py \
  --pdf-dir ../mib-doc-challenge/data/train \
  --out /tmp/train-evidence.jsonl
./.venv/bin/python scripts/replay_evidence.py \
  --cache /tmp/train-evidence.jsonl \
  --out /tmp/predictions.jsonl
./.venv/bin/python scripts/fit_runtime_posteriors.py \
  --cache /tmp/train-evidence.jsonl \
  --labels ../mib-doc-challenge/data/train_labels.csv

The cache contains only the pipeline's trusted, post-quarantine Packet representation. It is a versioned development artifact; the production CLI and container always ingest the supplied PDFs directly.

make score builds the linux/amd64 image, runs it with no network and a read-only root under the published resource limits, validates the output, and scores it with the public evaluator.

Repository layout

mib/            runtime pipeline
rules/          policy cascade and fitted posteriors
run.sh          container entrypoint
Dockerfile      offline linux/amd64 image
scripts/        profiling and fitting tools
tests/          unit and adversarial regression tests
docs/RECON.md   measured corpus and policy facts

Only mib/, rules/, run.sh, and their pinned runtime dependencies are copied into the image. No validation answers, case-id lookup tables, private data, or absolute development paths are included.