Skip to content

v0.3.0 — The OCR rung, and telling the truth in the docs

Choose a tag to compare

@drpwchen drpwchen released this 30 Jul 15:39

Where 0.2.0 was about table fidelity, this one is about the two places the project was
quietly asking users to trust something that wasn't there: an OCR rung whose central
component was never shipped, and documentation describing behaviour the code did not have.
Both were found the same way — by an outside user's agent following the docs literally and
burning hours on it (#3).

Added

  • A reference OCR adapter shipsconverter/surya_adapter.py, targeting Surya 0.22.x, plus
    docs/surya-adapter.md (#4).
    SURYA_ADAPTER had been a first-class config value, required for surya_available(), executed
    as a subprocess — and the file it pointed at existed only on the author's machine, with no
    published interface. An agent following the setup guide reasonably assumed it was part of the
    repo, could not find it, and reverse-engineered one against the removed surya.ocr API. The doc
    now publishes the contract (JSON Lines on stdout, fixture + blocks[].text + required
    blocks[].bbox, one line per image even when blank, logging on stderr, non-zero exit fails the
    batch) so any engine can take that rung, and states plainly why an adapter written for Surya
    0.17 dies on 0.22: surya.ocr is gone, FoundationPredictor became SuryaInferenceManager,
    and blocks now carry html rather than flat text.
  • Documented, tested memory caps for the OCR inference server. Surya 0.22 serves its VLM
    behind llama.cpp or vllm, and llama.cpp sizes its KV cache as
    parallel × ctx_per_slot — defaulting to 8 × 12288 = 98304 tokens for a model whose weights
    are ~1.4 GB. That cache, not the model, is what became a reported 48 GB run for one user. The
    doc ships a launch command with the caps in it; measured peak RSS with them was 3121 MB.
  • OCR output QC gate — the OCR rung now fails loud like every other rung. surya_ocr_pdf()
    counts unparseable adapter lines, images with no answer, and empty pages, and raises instead
    of writing markdown
    when the empty-page ratio exceeds T2N_OCR_EMPTY_PAGE_MAX (0.35) or mean
    characters per page falls below T2N_OCR_MIN_CHARS_PER_PAGE (200). Previously a JSON parse
    failure was silently continued and a near-empty book reported success in KB — the shape of a
    real incident where a scanned book produced 20 KB where ~1.6 MB was expected (~25 chars/page).
    Thresholds are calibrated against two real scanned references (689 and 788 pages: empty ratios
    0.054 / 0.003, mean 1791 / 2005 chars per page), so they clear legitimate blank and plate pages
    by roughly an order of magnitude. Per-book counters are surfaced in the batch report.

Fixed

  • Two-column reading order on OCR'd pages — the OCR path sorted blocks by (y0, x0), which
    walks across the gutter and back on every band of a two-column page, interleaving the columns.
    Every character is present and individually correct, so no downstream check could see it — the
    same defect the fitz path already carried a dedicated column sort for. Both paths now share one
    column_order_boxes() (T2N_COLUMN_SORT=0 restores the old sort on both). Byte-identity of the
    fitz path across two full books was verified as part of the same re-conversion run used for #5.
  • DOCLING_DEVICE defaults to auto instead of a hardcoded "cuda", resolving CUDA → Apple
    Silicon MPS → CPU inside the worker. A CPU-only machine no longer asks Docling for CUDA, and
    mps is now reachable at all (previously anything that wasn't the literal "cuda" mapped to
    CPU). Resolution logic is unit-tested without torch or a GPU; the real MPS path is unverified
    — there is no Apple Silicon in the development environment.
  • pdfplumber page cache is released after each page (#5).
    convert_pdf() opened one pdfplumber.PDF for a whole book and reached into plumber.pages[i]
    per page. PDF.pages holds every materialized Page for the object's lifetime, and each Page
    caches its chars / edges / rects on first use, so peak memory grew roughly linearly with the
    number of pages the table gate let through — nothing released them until close() at the end of
    the book. Every page is read exactly once, so flush_cache() + close() after its single use
    cannot change output, and byte-identity was verified on two books (a 1297-page table-dense
    reference, 627 tables, and a 638-page ordinary one): 23/23 output files identical each, before
    and after. Measured peak RSS: 6899 MB → 803 MB on the dense book (8.6x), 6209 MB → 133 MB
    on the ordinary one (46.6x), and the curve flattens instead of climbing to the last page.
    The ordinary book is the more interesting number: the issue predicted table-sparse books would
    barely leak because the gate skips them, but the gate admits far more pages than actually yield
    tables (638 pages, 12 tables extracted, still 6.2 GB), so the leak was never confined to
    table-dense books.

Changed

  • Documentation audited against the code, and corrected where it disagreed. Each of these
    was somewhere a reader could act on the docs and get a different result than the repo delivers:
    semantic search was described as if an indexer shipped (it does not — post_convert.py --index
    prints [skip] and returns success without INDEXER_SCRIPT, which is now stated, with the
    companion repo vault-search named as the thing to
    point it at); requirements.txt was said to cover the semantic-search stack (lines 7-16 are all
    comments); the figure-remap contract was documented with a match_method key that the
    validator actively rejects (the real key is match_quality, and qc_degraded / qc_skipped
    were undocumented); docs used a textbook-md/ output directory that is really OUTPUT_DIR
    (default ./output); architecture.md pointed at a section of itself that does not exist; and
    scan_fix_negatives.py read OLLAMA_VISION_MODEL while the rest of the repo uses the T2N_
    namespace (now T2N_OLLAMA_VISION_MODEL, with the old name kept as a fallback so existing
    environments don't silently switch models). Also corrected: the skill claimed a strict figure
    hard_fail exits 2 (it exits 1), and that chapter splitting is not attempted on OCR'd books
    (it is, best-effort).
  • README no longer claims per-page OCR engine selection (both variants). The detection
    signals are per-page; the routing decision is per-book — one trip of the check sends the whole
    PDF to OCR. Per-page routing is future work and is now labelled as such rather than described
    as shipped.
  • Setup guide no longer provisions OCR up front (AGENTS.md, #3).
    Step 1 asked whether the user had a GPU and pointed at the OCR ladder before a single page had
    been converted, which reads to a coding agent as "install the OCR stack now." It is an exception
    path: OCR routing exists only in the --batch-dir code path, and the single-file path never
    invokes it at all. A first-time user's agent installed Surya, hit the missing adapter (#4),
    wrote its own against the removed surya.ocr API, chased that into a local VLM inference server,
    and exhausted 48 GB of RAM — on a born-digital PDF that converted correctly with no OCR at all.
    The GPU question is gone from Step 1 and OCR now lives in a new Step 4.5, entered only when
    Step 4's output is actually garbled or empty, with a diagnose-before-installing checklist.
  • Related-projects section now links note-supplement
    (both README variants). It covers the direction this pipeline deliberately does not: merging new
    source material into notes that already exist, where the risk is not missing content but silently
    overwriting content the existing note already got right.