v1.0.0-rc.33
Pre-release
Pre-release
Added
- Stable node ids on the document tree.
DocumentNode.idis now populated with a
deterministic, content-derived id (stable across extractions), and each chunk carries
metadata.nodeIdslinking it back to the nodes it was derived from. Both are exposed as plain
strings across all language bindings. (#1296) - Per-page chunk coordinates (
pageSpans). Each chunk now carriesmetadata.pageSpans, a
list of{page, bbox}entries — one per page the chunk overlaps, in page order — for viewer
highlighting. The bounding box is the union of that page's body-layer node boxes within the
chunk (omitted when unavailable). Empty (and omitted from the wire format) when page provenance
is unavailable. (#1295) - Stable table ids, headers, and opt-in anchors.
TablegainstableId(a deterministic,
reproducible id assigned in document order across the final emitted table set) andcolumns
(the header row, so a fragment is interpretable on its own). A newtable_anchorsextraction
option (default off) emits a[TABLE:{tableId}]marker before each rendered table so consumers
can reconcile the markdown incontentwith the structuredtables[]entries. (#1297) - Configurable multi-label chunk classification. New
ChunkClassificationConfig(label
definitions, anLlmConfig,batch_size,max_concurrency), set via
ExtractionConfig.chunk_classification, classifies each chunk with zero, one, or many labels via
bounded-concurrency LLM batches and writes them tometadata.classifications
([{label, confidence?}]). Opt-in — absent config performs no classification and makes no LLM
calls. The chunk-level analogue of page classification; requires theclassificationfeature. (#1255) - Automatic OCR of undecodable text layers. Pages whose embedded text decodes mostly into the
Unicode Private Use Area, replacement, or control-character garbage (e.g. Identity-H fonts with no
ToUnicode) are now routed to OCR like scanned pages, controlled by
OcrQualityThresholds.min_undecodable_ratio(default 0.5) and gated by a 64-character floor so
occasional symbols never trip it. (#1254)
Fixed
- Concatenated-PDF extraction audit. A cluster of structure/reading-order defects surfaced by an
audit of concatenated legal PDFs: duplicatetables[]entries and double-counted table counts are
deduplicated (#1288); fragmented per-row tables are stitched back into one table with a propagated
header (#1290); bullet lists are no longer flattened into paragraphs or shredded into
word-per-cell tables (#1301); spurious intra-word spaces in native text ("T ower") are removed
via geometry-aware span joining (#1291); markdown escaping is consistent with an
escape_markdownopt-out (#1292); OCR'd pages no longer report contradictoryisBlank(#1293);
chunkfirstPage/lastPageprovenance is restored on long PDFs (#1294); and heading context no
longer leaks across document boundaries (#1289). - Container users can enable page classification via a
classificationCLI feature carried into
theall/container builds. (#1298) - CLI batch resource limits and benchmark isolation.
extractandbatchnow support
--no-config-discovery, and their Tokio runtime honors--max-threads. The benchmark harness
uses both controls, records batch subprocess overhead from the reported batch total, and keeps
user configuration and verbose PDF logs from contaminating measurements. - Linux glibc artifacts no longer require a newer glibc than they advertise. The prebuilt ONNX Runtime that was statically linked into the Python wheel and the Go/C FFI, Java, and Elixir artifacts is compiled against glibc ≥ 2.38 and pulled unversioned
__isoc23_*and__libc_single_threadedsymbols into the binary. Because the manylinux tag only reflects versioned glibc requirements, these installed everywhere but crashed at import/load on glibc 2.28–2.37 (undefined symbol: __isoc23_strtoll). They now link Microsoft's ONNX Runtime (glibc-clean, floors at 2.27) — the same runtime the C#, Node, and Docker builds already use — and ship it beside the binary, so the symbols are gone and the floor is honest. A CI check now fails any glibc artifact that references too-new symbols or omits the bundled runtime. musl, macOS, and Windows are unaffected.
Changed
- Dependencies refreshed across all languages. Rust, Python, Node, Ruby, Java, and Dart
dependency versions were bumped to their latest compatible releases.