Skip to content

PageLedger 0.7.0

Choose a tag to compare

@peterbussch peterbussch released this 02 Oct 14:55
· 225 commits to main since this release
11fde5a

PageLedger 0.7.0 adds an escalation ladder for document jobs: pages climb from local OCR to a vision model on more than warning holds, the model's reading is checked word by word against the OCR, and the differences that remain are settled against the page image, with every reading, span and decision kept as a verifiable record.

  • Climb. processing.escalate_on sends a page to the image stage when a warning holds it (as before), when its share of known words is low (rough, with the Russian lexicon from pageledger[ru]), when clean engines disagree (disagreement), or for every page (always). Each trigger is recorded on the page with its evidence, and the report shows the work done at each stage.
  • Contest. processing.contest compares a model's reading with the literal engine's and keeps each difference as a span. Look-alike letters and RapidOCR's known glyph confusions are settled by rule; a non-word those confusions don't explain stays open, because it may be the printer's misprint. Pages with open spans are held as contested.
  • Settle. pageledger adjudicate --packets writes each contested page's open spans and image, blind to which engine read what; --decisions records the answers as receipts bound to those spans. A page with no open span becomes adjudicated_text, and its edition is what transcripts and exports carry. verify-job rebuilds the spans and editions from their receipts.
  • Measured. On a hand-checked Russian article, RapidOCR differed from the checked text in 91 words, the model's reading in 17, and the adjudicated edition in 4. Across two hand-checked documents (61 pages) the rules settled 178 differences with no errors, and the spans covered 306 of the 319 known corrections.
  • Also new: a foreign_script_characters warning, reading_order: columns for the rapidocr adapter, and doctor --config reporting the keys a config names through env_key.
  • Fixed (from the unpublished 0.6.1): a scanned page with no text layer no longer waits for a person once a non-generative engine reads it without concern. New jobs record hold_policy: "0.6.1".

Validation: the full local suite passed (1,529 tests; 1,536 with the ru extra) with Ruff and mypy clean; wheel and source-package inventories passed; both maintained tutorials passed against isolated installations of both packages. New artifact fields are optional, so jobs written by 0.6.0 verify unchanged.

Install with pip install pageledger==0.7.0, or pip install "pageledger[pdf]==0.7.0" for PDFs; pageledger[rapidocr] adds the RapidOCR engine and pageledger[ru] the Russian lexicon. Scanned PDFs also require separately installed OCR tools.

The attached distributions and SHA256SUMS are the exact artifacts from the protected PyPI publication workflow (run 37019621280).

5a9d5965361af5ab1dfb2ece427e833e6889de7e384f85cab62a8e20dc179b12  pageledger-0.7.0-py3-none-any.whl
1de0500bf774db22b9a7f753e8217c5a39e24b400edceafbf8343d00132c3b0c  pageledger-0.7.0.tar.gz