Extraction Quality

Extraction quality

What the extracted markdown is actually like, and what the quality instruments can and cannot see. The pipeline that produces it is Extraction-Pipeline.

Corpus state

Measured on the full-corpus run, 2026-08-02.

Throughput	~1,400 docs/hr at CONVERTERS=4 / TEXT_WORKERS=8 on a 64 GB / 16 vCPU host
Path split	34,153 text / 17,286 OCR; 7,628 carry no provenance
Documents with chunks	53,109 of 60,578 (87.7%)
Chunks in Cosmos	995,316 — 18.7 per document
Genuinely unextractable	~106 (0.18%)
Source object 404s	5,802
Extracted but textless	2,039 large-format sheets, recovered by tiling
Ingest throughput	~23,900 documents/hour at UPLOADERS=4

Word-joining

src/scripts/audit-chunk-quality.js scored 1,299 chunks across 400 documents on 2026-07-31: 71.8% clean, 12.4% marginal, 15.8% garbage — an upper bound, since it counts table-of-contents pages as garbage. OCR word-salad was 3 chunks, 0.23%.

The defect the heuristics are structurally unable to see is word-joining:

Laren: Iwant tovoicemyopposition to theKitimat-Summit
IN THE MATTER OF ENVIRONMENTALASSESSMENT CERTIFICATE

Every glued fragment is alphabetic and pronounceable, so it scores clean on every cheap metric. A user searching the words they can see on the page gets nothing, because the index holds one token where the page has four. This is the inverse of the Tum ble r Ridge character-spacing artefact, and it defeats heuristics for the same reason.

Rate per 1,000 word tokens, 400 documents per stratum, measured 2026-08-03:

Stratum	alphabetic token ≥16 chars	internal lower→Upper
text	0.44	2.70
ocr, docling-parse backend	10.23	17.45
ocr, pypdfium backend	12.37	18.05
ocr, tiled	12.76	37.57

23–29× more frequent on the OCR path. The text figure is the detector's false-positive floor, not real joining — its own worst example is a legitimate English word. The backend choice does not change it, and tiling roughly doubles the camel rate, consistent with map lettering.

Word-joining is only 3 of 32 retrieval misses, 9%. It is the dominant defect in the extracted text and not the dominant cause of missed searches — see Search-and-Retrieval, where that was measured directly. Two claims that read as one.

An intake cleaner does not address it. Stripping <!-- image --> and dropping separator chunks is unrelated to a missing space inside a token; the fix belongs in extraction or in a decompounding step at index time, and either is a new component rather than a filter.

What the instruments get wrong

src/text-quality.js holds the heuristics. Three of them would aim a fix at the wrong target unless configured against the corpus:

• mostly-separator-furniture fires on the dot leaders of a table of contents inside a perfectly clean report. It is gated on letter density as well as separator share, because a real contents page runs a high separator ratio while still carrying findable section titles.
• A hydrology data table scores identically to OCR debris (vowelless 0.27 against 0.24) unless identifiers like PH12-3-3 and 3E-06 are excluded. Cheap metrics cannot separate the two on any single signal.
• Tabular detection that only understands pipe tables flags a table of earthquake records. Whitespace columns are still columns, so both forms are detected.

The 400 audited documents are taken in Cosmos scan order, which skews heavily toward Site C hearing exhibits. It is a sample of that cluster, not of the corpus.

<!-- image --> is docling's own placeholder and image_export_mode cannot switch it off — it admits only placeholder, embedded, referenced.

Some "PDFs" are saved 404 pages

PDFium: Data format error is worth one check before assuming a corrupt PDF: fetch the object and look at the first five bytes. At least one document (Vol 8 - Map P7 Mineral Titles and Reserves) stores a 268-byte HTML error page written by the original ETL when the legacy EPIC system answered 404.

fileExt: pdf | mimeType: application/pdf | fileSize: 268
<!DOCTYPE HTML ...><title>404 Not Found</title>

The metadata still claims application/pdf. The real file is gone from the source system, so recording an extraction error is correct — there is nothing to extract and no engine change recovers it. Rate: 2 of 9,034 documents downloaded, about 0.02%, so on the order of a dozen corpus-wide. fileSize under ~1 KB on a pdf is the cheap detector. The document count therefore slightly overstates retrievable content.

Tiling large-format sheets

2,039 documents extract to nothing but <!-- image --> placeholders — in the index and unfindable by content, which is worse than noisy text. These are D-size drawings. Docling normalises a page image to a fixed size before OCR, which puts 6-point labels on a large sheet below RapidOCR's detection floor.

Tiling cuts each page into a TILE_GRID × TILE_GRID (3×3) grid with TILE_OVERLAP 6% overlap, bounded by TILE_MAX_PIXELS (60M) and TILE_MAX_PAGES (20). Of the 2,039, 1,855 (91%) hold real text afterwards — median 299 real characters, p75 552, p90 1,017. The residue is 184.

Two things to know about the result:

tiled: true means tiling beat the empty pass, not that the text is usable. 83 of the 184 residue documents carry it. Tiled text is rough where map lettering sits at an angle (Barrowsources, Offsite constnk source Of).

contentPageCount cannot see this recovery. 1,874 of the 2,039 still read exactly 1 chunk, because ~300 characters is one chunk either way; what changed is that the chunk holds text instead of a 14-character placeholder. Measure the markdown or run a query, not the count.

LOW_YIELD_MIN_BYTES defaults to 200,000. Setting it to 0 breaks worker.py --selfcheck, which asserts a tiny file must not be sent to the GPU. That is a correct invariant for the general corpus, so the override belongs on a hand-built work list of affected documents and never on a corpus run.

See also

• Extraction-Pipeline — how a stored PDF becomes chunks
• Search-and-Retrieval — the retrieval scorecard, and which defects actually cost a search