Added
-
Added per-page OCR confidence to
PageContent.ocr_confidence, reported as a
PageOcrConfidence { score, word_count, backend }
(#1568). The field is absent for pages that
were not OCR'd.scoreis populated only for backends whose confidence is a calibrated
legibility scale (normalised to0.0..=1.0) and isNonefor uncalibrated ones, so a page
OCR'd without a comparable score is still distinguishable from a page nobody scored. It is
reported alongsideword_countbecause noise filtering runs before the score is computed: a
high score over very few surviving words does not mean the page read well. -
Added HWPX (Hangul Word Processor XML) extraction to the WebAssembly package.
unhwp
target-gates its ZIP reader to a deflate-only, LZMA-free build underwasm32, so the
native-C dependency that previously kepthwpxoffwasm-targetdoes not apply there. -
Added diagram recovery from flat OpenDocument drawings (
.fodg), including content-based
detection of theapplication/vnd.oasis.opendocument.graphics-flat-xmlMIME type. Connectors
name their endpoints outright, so the recovered graph is exact rather than inferred from
geometry (#1545 corpus fixture). -
Added structural extraction for MyST Markdown syntax and MyST text notebooks, including saved
inline{eval}values in Jupyter markdown cells
(#1538). -
Added extraction of Jupytext percent- and light-format notebook scripts, including
text/x-python,text/x-r-source, andtext/x-juliaMIME aliases
(#1538). -
Added bounded, cancellable SQLite and GeoPackage table extraction with schema-based GeoPackage
detection,.sqlite3and.gpkxfilename support, and defensive handling for untrusted
databases (#1510). -
Added configurable MIME inference policies for preferring content signatures, trusting supported filename
extensions, or ignoring extensions, with per-input overrides (#1509). -
Added native KML and GeoJSON extraction with canonical MIME routing (#1508).
-
Added Rust
SUPPORTED_FORMAT_COUNTandSUPPORTED_EXTENSION_COUNTconstants derived from the
MIME registry, plus automated synchronization for published format-count claims (#1511). -
Added reusable Rust PDF render sessions for querying page counts and rendering multiple pages
without reopening the document (#1485). -
Added cooperative cancellation for single and batch extraction (#1476).
-
Added dynamic system linking for Tesseract and Leptonica through the
tesseract-dynamicfeature
(#1407). -
Added managed Azure AD, Google Vertex AI, and AWS STS credential providers, with credential values
redacted from debug output. -
Added reasoning-effort, provider-specific request-body, and Bedrock configuration for LLM
extraction. -
Added
xberg doctorand the Rustdoctor()API for validating configuration and probing every
compiled OCR, VLM, layout, table, formula-recognition, and cache capability without downloading
models or contacting remote providers.xberg doctor --cleanremoves stray files only from
Xberg-owned caches (#1347). -
Added the Sceptre EasyOCR Gen2 backend for desktop, mobile, and WebAssembly.
-
Added sparse and late-interaction embeddings to chunk output.
-
Added a Prometheus
/metricsendpoint to the API server (#1391). -
Added explicit CSV delimiters and comment-line prefixes through
CsvOptions. -
Added
xberg tree-sittercommands for downloading, listing, and cleaning language assets, with
optional configuration-file loading. -
Added VLM extraction for complex PDF regions and LaTeX formula extraction from VLM OCR.
-
Added structural AsciiDoc and WebVTT extraction.
-
Added Docling DocTags input and output, including tables and page geometry (#1383).
-
Added formula recognition for rasterized pages and exposed formulas consistently across extracted
formats (#1385). -
Added JATS, EPUB, ODT, and ODP MathML-to-LaTeX conversion.
-
Added deterministic diagram recovery from SVG and PDF sources with Graphviz DOT output (#579).
-
Added
SecurityLimits.max_pagesfor PDF, presentations, Keynote, and multi-frame TIFF documents
(#1451). -
Added explicit PDF backend selection through
PdfConfig.backendand--pdf-backend(#1448). -
Added musllinux Python wheels and a Windows x86_64 Ruby gem.
-
Added PDF and HTML extraction plus layout and transcription types to the WebAssembly package.
-
Added
--ocr-no-cacheto bypass the Tesseract result cache. -
Added
ContentFilterConfig.include_footnotesfor retaining footnotes classified as page furniture. -
Added a public
render_heading_breadcrumbhelper for retrieval-oriented chunk content (#1393). -
Added structured-output merge, citation, and vision-fallback helpers for Rust embedders.
-
Added a Tower-compatible extraction service, request type, and builder for Rust applications.
-
Added typed configuration for TrOCR, PaddleOCR-VL, GLM-OCR, and DeepSeek-OCR backends.
-
Added
classify_chunks_ownedfor classifying and returning an owned document. -
Exposed chunk-classification and LLM concurrency, provider, cache, budget, and rate-limit configuration
types at the Rust crate root. -
Added
OcrConfig::security_limits.ExtractionConfig::security_limitsis now threaded through to
every OCR route — embedded images, Tesseract, PaddleOCR, and scanned PDF pages — instead of each
route decoding images under a hardcodedSecurityLimits::default()
(#1554). -
Added
detected_language_confidences, carrying each detected language's confidence, proportion,
script, and reliability alongside the existingdetected_languagescodes, so a document that is
95% English and 5% French is distinguishable from an even mix
(#261). The existing field keeps its type and ordering. -
DOCX reviewer comments now emit their own
NodeContent::Commentnode instead of riding the
footnote reference and definition machinery, so consumers can tell a comment from a footnote. -
PDF annotations now preserve their subtype (Ink, Square, Circle, Polygon, PolyLine, Line, Squiggly,
Caret, FileAttachment, Sound, Movie) instead of collapsing toOther, carry author, modification
date, colour, subject, and QuadPoints, recover the text a Highlight marks, and are emitted by the
Markdown, Djot, plain, HTML, and JSON renderers — previously no renderer emitted annotations at all
(#63). -
PDF extraction now reads image alt text from the structure tree, falls back to XMP for title,
author, and subject when the Info dictionary is empty, surfaces/PageLabels(roman-numeral front
matter, per-section numbering) throughmetadata.additional, excludes content on optional-content
layers that are off by default, and renders filled AcroForm values. Unencodable images, annotation
failures, and form failures now emit aProcessingWarninginstead of being dropped at log level
(#62, #71). -
The OOXML
DocSecuritybit field is decoded into named protection flags onMetadata.additional
for DOCX, XLSX, and PPTX, so a password-protected or read-only-recommended document is
distinguishable from an unrestricted one. -
Added PaddleOCR on the tract backend, so classical PaddleOCR (DBNet, CRNN, AngleNet) is available on
wasm32and the Android x86_64 emulator, where ONNX Runtime cannot link. -
Added
top_p,stop,seed,presence_penalty, andfrequency_penaltytoLlmConfig, validated
and applied to every outgoing request. They were previously accepted by every config file and
language binding and then dropped before reaching a provider. -
Added
LlmConfig.max_concurrencyto bound VLM OCR and image-captioning requests in flight
independently ofConcurrencyConfig.max_threads, which represents local CPU capacity
(#1453). -
Every error variant now carries a stable FFI error code, so typed error handling works in the C-ABI
bindings;errors.Is(err, ErrOcr)in Go, Java'scheckLastErrorswitch, and Zig's error set
previously collapsed all variants to a single unknown constant. -
Exposed
html_to_markdown_rs::ConversionOptionsat the Rust crate root, so callers configuring
ExtractionConfig::html_optionsno longer need a direct dependency on the upstream crate, and made
DocumentNode's text and node-type accessors public soDocumentStructure.nodescan be read as
documented. -
Added
FormatMetadata::html(), returning the HTML metadata when the variant isHtml, matching the
accessors already exposed for the other formats. -
Added an opt-in Pdfium PDF extraction backend behind the
pdf-pdfiumfeature, selectable with
PdfConfig.backendor--pdf-backend pdfium, providing page count, per-page text, and Info
dictionary metadata. Its scope is deliberately narrower than the native engine — no table detection,
layout integration, form fields, or OCR fallback — and every result carries aProcessingWarning
naming the gap. The feature is not part offull, so it reaches source builders only. -
Added a Scoop manifest published to the
xberg-io/scoop-bucketon release, so the Windows CLI can be
installed withscoop install xberg. -
Extraction now reports a
ProcessingWarningwhen a document decodes lossily or degrades silently.
Decode provenance is captured before mojibake cleanup strips the replacement characters that used to
be the only evidence, and archive, AsciiDoc, WebVTT, XML, and plain-text extraction warn on replaced
characters. Unresolved ODT image hrefs, unparseablestyles.xml, collapsed repeated table cells,
skipped LaTeX, Typst, RST, and Org includes, OPML without a body, links past the per-document URI
cap, truncated XML, and words Tesseract failed to extract now warn instead of failing silently
(#171, #133). -
A PDF page whose raster render comes back blank now falls back to OCR'ing the page's embedded image
XObjects, and that recovery preserves the tables, formulas, LLM usage records, and image
preprocessing metadata the backend produced instead of keeping only the text, with every recovered
payload accounted againstsecurity_limits.
Changed
-
Breaking (Python binding):
ExtractionConfigandDoctorReportare now frozen dataclasses
rather thanTypedDicts, matching the 121 option types that were already dataclasses. Passing a
plaindictor a JSON string asconfigstill works —extract()coerces both — but an
ExtractionConfigobject no longer supports mapping operations, soconfig.get("chunking")and
config["chunking"] = ...now raiseAttributeError/TypeError, and the instance is immutable.
Build a modified config withdataclasses.replace(config, chunking=...). -
PDF parsing no longer reports recoverable input at WARN. A missing embedded font, an object
outside the xref table, an unreadable CFF version, and a reading-order fallback are ordinary
properties of real PDFs rather than conditions an operator can act on; they are now TRACE (or
DEBUG for strategy fallbacks), and each document emits a single DEBUG summary on the
xberg_native_pdf::recoverytarget carrying the totals instead of one event per occurrence.
Measured over a 4,000-document corpus this removed 4,012,488 of 4,014,206 log events, against
which 44 genuine parse failures had been sitting at a ratio of about 1 in 91,000. ERROR
behaviour is unchanged — it already corresponded one to one with documents that failed
(#1547). -
Breaking (Rust source):
validate_mime_typeno longer accepts any value with animage/
prefix. It now parses the MIME type and requires exact membership in the supported-format
registry, so unregistered vendor image subtypes such asimage/x-custom-formatare rejected as
UnsupportedFormatinstead of validating (#1511). -
Per-page OCR recognition-noise detail (fragmented-word ratio, word count, mean confidence) now
reaches the page accept/reject decision and is emitted atDEBUGinstead of being discarded one
frame earlier. No threshold is gated on it yet; the blended stage score alone cannot discriminate
noise pages. -
Breaking (Rust source):
ExtractionConfigaddsapply_notebook_cell_tags. Notebook
extraction now honors MyST and Jupyter Book remove/hide cell tags by default; set the field to
falseto retain all saved cell content
(#1538). -
Breaking (Rust source):
OcrQualityThresholdsaddsdiscard_suspected_ocr_noise; exhaustive
struct literals must set the field or use..Default::default(). -
Breaking: configuration deserialization now rejects unknown fields in nested Xberg
configuration tables instead of silently ignoring misspelled settings. -
Breaking: PDF backend configuration now uses
"native"andPdfBackend::Nativeinstead of
"pdf_oxide"andPdfBackend::PdfOxide. Update explicit configuration values; the default is
unchanged. -
Breaking:
EmbeddingModelType::LlmandRerankerModelType::Llmnow carry their model name in
the enum variant. -
Breaking:
Formula.bboxandFormula.pageare optional so formulas from formats without page
geometry can be represented. -
Breaking: unknown multipart fields on extraction endpoints now return an error instead of being
ignored. -
Chunk
contentnow contains the exact source span; heading breadcrumbs are available separately. -
The CLI
allfeature now includes audio transcription. -
security_limits.max_pagesnow applies to presentations, Keynote, and multi-frame TIFF as well as
PDF. -
create_client_with_credential_providernow returnsManagedClient, and an LLM concurrency limit
of zero is rejected. -
Native PDF pages now expose their final per-page reading order.
-
WebVTT cue timing is optional for blocks without a timing line.
-
OpenDocument packages without
content.xmlnow return an extraction error. -
CLI text output now includes the extraction envelope with warnings, timings, and metadata.
-
CLI JSON output now reports peak resident memory.
-
Windows builds now include the same supported feature set as other desktop builds.
-
Breaking: Rust element identifiers now use
Stringdirectly; theElementIdwrapper has been
removed. -
Breaking: Public tuple fields for ranges, coordinates, dimensions, links, code blocks, and attributes now
use named Rust structs and serialize as JSON objects. Legacy positional JSON arrays are still accepted when
parsing, so payloads written by 1.0.x keep deserializing, but they are no longer emitted. -
Breaking: removed the duplicate
xberg::llm::region_extractor::RegionKind; importxberg::RegionKind
instead. -
Parsing and configuration deserialization now reject invalid region, redaction, and reranker values.
-
Corrected and expanded installation, CLI, configuration, extraction, migration, integration, and
cross-language API documentation. -
Corrected canonical MIME and extension routing for DBF, YAML, reStructuredText, Org, Typst,
XHTML, Djot, JPEG 2000, HEIC/HEIF, MP4, and MPEG inputs. -
GeoJSON extraction now returns a bounded aggregate summary by default, including feature,
geometry, property-key, position, and bounds metadata. Set
geojson.include_full_coordinates = trueto retain the complete document and coordinate arrays. -
quality_scorenow explicitly measures the cleanliness and readability of retained text, not
extraction completeness; inspectprocessing_warningsfor known partial or degraded results. -
The default
security_limits.max_table_cellsremains 100,000 aggregate cells per document;
limit errors now explain how to raise it for trusted inputs or reduce the source table. -
TesseractConfig.language_model_ngram_onnow defaults totrueon both the PDF and standalone
image OCR paths. Tesseract previously applied no penalty to output that does not look like a word of
the target language, the dominant failure mode on scanned line art. Set the field tofalseto
restore the previous behaviour. -
Tesseract Markdown-format OCR now drops hOCR lines whose dictionary-checkable words are more than
60% invalid, removing recognition noise such asOWATS DNDEVETwhile keeping labels likeEXHIBIT
andLEGEND. A line needs at least two checkable words to be scored, and the removed-line count is
reported as aProcessingWarning. -
Undecodable-text OCR routing is now decided per page rather than for the whole document, so a single
unreadable page no longer sends every page of a PDF through OCR and discards good native text. The
previous document-wide fallback still applies when page boundaries are unavailable or inconsistent. -
With
max_threadsunset the thread budget ismin(num_cpus, 8)and now ceilings Rayon, ONNX Runtime
intra-op threads, and batch workers alike. A cgroup CPU quota is honoured in place of the hardcoded 8
where one exists, and a host with more than 8 cores and nomax_threadsis warned once per process
(#1392). -
PaddleOCR inference now uses the resolved thread budget instead of a hardcoded single thread. The
session is serialized behind a mutex, so exactly one worker runs and can claim the whole budget
without oversubscribing.
Removed
- Breaking: removed the inert
ChunkingConfig::prepend_heading_context,breadcrumb_target,
BreadcrumbTarget, and corresponding CLI and environment options; use chunk metadata or
render_heading_breadcrumbwhen a retrieval index needs headings inline. - Breaking: removed
OutputFormat::Structured; usePlainfor unrendered content orJsonfor a structured
content tree. - Breaking: removed
ExtractedDocument.formatted_contentfrom language bindings; usecontent
or select the desired output format during extraction. - Removed advertised support for troff, mdoc, POD, and DokuWiki because they did not have structural
extractors. - Removed fabricated OCR
script_nameandscript_confidencevalues. - Removed the unused public
LanguageRegistry,BatchProcessor, object-pooling APIs, and unused
tree-sitter re-exports. - Removed the nonfunctional
wasm-threadsfeature. - Removed PDF writing, editing, building, and XFA conversion APIs from the native PDF crate; read-only
XFA analysis remains available. - Breaking: removed the inert
Enginestructured-policy, preset-resolver, LLM-client, and model-provider
injection methods. - Breaking: removed the inert transcription field from
EnrichmentConfig; configure transcription during
extraction instead. - Breaking: embedding, reranking, sparse-embedding, late-interaction, and preset APIs are now exposed only
when their required features are enabled. - Breaking:
core::batch_mode,core::formats, andcore::ioare now crate-private, and the public
DocumentStructureBuilderhas been removed.
Fixed
-
Fixed the Windows Ruby gem failing to build.
xberg-libwpd's build script chose its zlib by
operating system alone, so the gem's MinGW/UCRT toolchain was handed vcpkg's MSVC-built
x64-windows-static-mdarchive and the link died withcorrupt .drectve/ld returned 5. The
vcpkg path is now taken only for genuinely MSVC targets; every other target links the static
zliblibz-sysalready builds from source. -
Fixed
XbergLoaderignoring chunking and per-page splitting whenever the LangChain
integration was given anExtractionConfigobject. Both settings were read only when the
config was adict, so afterExtractionConfigbecame a frozen dataclass the documented
ExtractionConfig(pages=PageConfig(extract_pages=True))andchunking=ChunkingConfig(...)
forms silently produced one Document per file instead of one per page or chunk. The config
is now read as an object or a mapping. -
Fixed a ruled troubleshooting page collapsing into one table, taking its section headings
down with it as cell text.split_rows_by_text_positionssubdivides a producer-drawn row
band by the Y positions of the text inside it, and since the #1555 fix a candidate split was
accepted only when EVERY resulting Y-cluster carried text in at least two columns, with the
rejection all-or-nothing for the band. A band that mixes multi-column data rows with
single-column lines -- a section heading, a lead-in, a wrapped continuation -- can never
satisfy that, so one such line vetoed the split for the whole band and every line inside it
became cell text. On one 56-page installation manual, six ~20 pt row bands became a single
522 pt table, the document went from 808 elements to 759, and four numbered headings
disappeared from the outline. The band is now split once at least two of its clusters are
independently evidenced, and each deficient cluster is resolved on its own terms: it folds
into the cluster above only when it introduces no column that cluster left empty, which is
the signature of a wrapped continuation. Anything else -- a heading, a lead-in -- stays a row
of its own, one cell wide, which is what such a line inside a ruled band actually is. Two
independently evidenced clusters are required rather than one because a single evidenced
cluster can be coincidence, which is precisely the #1555 case
(#1565). -
Fixed a word split across two touching PDF spans being rejoined with a space, so
prijs
extracted aspri js. The gap between the two spans measures 0.069 pt -- 0.008 em at 9 pt,
against a 2.5 pt space glyph -- on an identical baseline at an identical font size, so no gap
threshold produced the space:segments_need_spacereached one of its unconditional
return truebranches first.SegmentDatakeeps onlyis_bold/is_italic/is_monospace
and dropsfont_name, so a mid-word switch between two embedded subset fonts whose
/FontDescriptors disagree onForceBold,ItalicAngleorFixedPitchreads as a style
change carrying no geometric signal at all. That is why the defect never reproduced against
base-14 Helvetica, and why widening the gap to 2 pt changed nothing. A touching-spans guard
now runs before those branches: two segments on the same baseline, at the same font size,
with alphanumeric characters on both sides of the boundary and a gap under 0.025 em are one
word and are concatenated. The guard can only join, never split, and it never fires across an
explicitly drawn space. The table path needed the same test one stage earlier, in
segments_to_words, becauseHocrWordis integer-rounded and cannot represent a sub-point
gap by the time cell text is joined. Affects ordinary prose, not just tables: of 18 confirmed
cases, 14 wereNarrativeText, 3ListItemand 3Table
(#1566). -
Fixed PDF table reconstruction dropping early rows when data-start inference classified more
than two leading rows as headers. The two-row header cap is retained, but surplus inferred
header rows are now demoted to data in source order instead of being discarded
(#1558). -
Fixed native PDF top-to-bottom reading order splitting one visual table row at an absolute
3-point coordinate-band boundary, which could move an article number before its position and
fuse the two identifiers. Visual rows now use an anchored, font-scaled tolerance, reconstructed
lines restore left-to-right fragment order, and narrative assembly preserves a separator after
a severe geometric backtrack (#1560). -
Fixed PDF dehyphenation treating inline run/style boundaries as visual line wraps. Suspended
hyphens such asvracht- en verzendkostenare now preserved, while compounds genuinely split
across different baselines are still rejoined
(#1561). -
Fixed DOCX page attribution staying permanently low after Word omitted a rendered-page marker
between vertically stacked inline images. The parser now conservatively infers missing breaks
from each section's usable page height, including documents with different section geometries
(#1559). -
Fixed DOCX DrawingML and VML text boxes dropping XML and numeric character references such as
&and€from extracted text
(#1562). -
Fixed OCR image decoding ignoring the caller's configured
security_limits. Every OCR route —
embedded images, Tesseract, PaddleOCR, and scanned PDF pages — decoded raw image bytes under a
hardcodedSecurityLimits::default(), so raisingExtractionConfig::security_limitsto accept a
large scan still had it rejected at the OCR decode step. The configured limits now reach all four
routes, and PaddleOCR also honors a per-callbackend_options["security_limits"]override
(#1554). -
Fixed a drawn PDF table row with a wrapped cell being shattered into extra rows. Splitting a row
band by text Y-position now requires at least two columns to have independent text evidence for
every candidate row before splitting; a band where only one column wraps to a second line now
stays a single row (#1555). -
Fixed monospace font detection matching any font name containing "mono", misclassifying foundry
names such as "Monotype Corsiva" as a monospace font and skewing the word-spacing heuristic and
code-block detection that depend on it. "Monotype" is now excluded from the substring match, and
the PDF text run buffer's separate ad hoc monospace check was replaced with the same shared
helper. -
Fixed a standalone multi-line monospace paragraph not being recognized as a code block unless it
had a consecutive monospace neighbor paragraph. A lone paragraph that already carries two or more
monospace lines is now fenced as a code block on its own
(#1557). -
Fixed PDF text extraction silently corrupting ordinary text. A contextual ligature-repair pass
rewrote:totiand an uppercaseMbetween lowercase letters tottion every element of
every document, mangling identifiers, ratios, times, URLs, and units such asnM(for example
aMbbecameattib). The repair was introduced for European PDFs that encode ligature glyphs
at ASCII code points, but it was gated at the time on a per-font broken-CMap signal from
pdfium'shas_unicode_map_error(). That gate was lost when pdfium was removed as a backend and
was never ported to pdf_oxide, leaving the rewrite running unconditionally. Both substitutions
are removed; they can only return alongside a real document-level evidence gate
(#1556). -
Fixed optional fields in the Python and PHP bindings rejecting payloads that omit them.
The generated mirror structs lost their#[serde(default)]attributes, so deserializing a
document whose JSON left an optional field out failed instead of falling back to the default. -
Fixed legacy
.docheadings being guessed from line length rather than read from the document's
own styles. A paragraph styledheading 1..heading 9— directly or through a custom style
derived from one, such asTOC Heading— now becomes aHeadingat that level, instead of every
detected heading being a level 2. Documents that apply no heading style keep the previous
shape-based detection, because roughly half the test corpus styles its headings as boldNormal
and would otherwise lose every one; the choice is made per document, not per paragraph. A
heading-styled paragraph that is also list-bound stays aListItem, matching how the DOCX path
treatsw:numPr(#1553). -
Fixed legacy
.docautomatic list numbering being dropped entirely: a paragraph Word numbers
through its list tables arrived as prose, indistinguishable from an unnumbered sentence, while
the DOCX path emitted aListItemfor the same construct. Auto-numbered paragraphs now arrive
asListItems inside an ordered or bulleted list container, with their nesting depth, matching
the DOCX path. The number Word paints (1.1,a.) is still not rendered — recovering it needs
list-table counter state — so a document mixing automatic and hand-typed numbering shows the
typed numbers as text and the automatic ones as list structure
(#1550). -
Fixed legacy
.docelements being split on blank lines rather than on Word's paragraph marks,
which merged every pair of consecutive paragraphs not separated by a blank line into a single
element. One corpus letter returned its entire ten-paragraph body as one element. Word97 and
later documents now emit one element per Word paragraph, matching what the DOCX path does with
w:p. This changes element boundaries, counts and indices for most.docdocuments, and
alterscontentline spacing accordingly; consumers keying on element position will see the
difference. Word 6/95 documents and those falling back to contiguous text extraction keep the
previous blank-line behaviour, because they carry no paragraph properties to use. -
Fixed legacy
.docextraction readingfcClxfromFibRgFcLcb97pair 66 — an obsolete field
Word writes as zero — instead of pair 33, so the piece table was never walked for any document
and extraction always fell back to readingreserved5/reserved6, bytes [MS-DOC] requires a
reader to ignore. Where those bytes disagreed with the real text start, whole documents were
decoded as UTF-16LE and returned as glued CJK-looking code points; multi-piece and fast-saved
documents could not be assembled at all. Footnote, header/footer, comment, and text-box
subdocument text now also reaches the output for these files
(#1551). -
Fixed the Elixir NIF's vendored
Cargo.lock, shipped in the Hex package, pinning
tree-sitter-language-pack1.15.12 while the crate requires 1.16.1 — a source build of the NIF
with--lockedcould not resolve. This affects anyone whose platform has no precompiled
artifact and therefore builds from source. -
Fixed a DOCX table cell spanning several grid columns (
w:gridSpan) or rows (w:vMerge) being
returned once per covered column and again for every covered row, so a cell merged across 4
columns and 3 rows came back 12 times inresult.tables[].cells,result.tables[].markdown,
andresult.contentalike — a 39 KB document could extract to 232 KB. A merged/spanned cell's
text is now written once, at its origin, with the columns and rows it covers left blank. This
also fixes a DOCX header or footer table with a merged cell shifting every following cell one
column to the left (#1549). -
Fixed PDF render diagnostics matching a captured engine warning against a hardcoded message
substring to decide whether it meant a glyph actually failed to paint. The message it was built
to exclude no longer reaches this capture at all (it moved to TRACE under #1547), so the match
could only ever misfire: a future warning whose text happened to share that substring would have
been silently dropped instead of surfacing as aProcessingWarning. Every captured warning is
now reported (#1548). -
Fixed a PDF page that places a statistics table beside a prose column being emitted in
full-width Y order, which spliced the prose apart mid-sentence (more likely to be aged 35Female 51.5 ...) and welded the table's two label/value panels together on every row. The
table region is now emitted whole, in row order, ahead of the prose column, and a repeated
panel is emitted panel by panel
(#1545). -
Fixed PDF text coming back scrambled when a short
Tjrun sat between twoTJarrays: the run
was emitted at an earlier run's stale position and sorted into the wrong place, so
within a period ... after conclusionextracted aswincthin a period ... after co lusion.
Every text-showing boundary operator closed the pending run exceptTJ
(#1544). -
Fixed every image in a DOCX reporting
page_number1 regardless of the page it sits on. The page
was resolved by searching rendered Markdown for a per-image placeholder that is never written --
every drawing renders to the same link target -- so the lookup always missed. Page numbers now
come from the parsed element order (#1546). -
Fixed an author's hyphen being deleted when it fell at a line break, so
price-+determining
joined aspricedetermining. A hyphen written mid-line elsewhere in the same document is now
treated as evidence that the compound is real and its hyphen is kept. Compounds that appear only
broken, with no such occurrence anywhere in the document, are still joined without the hyphen
(#1543). -
Fixed OCR backends registered through
register_ocr_backendbeing rejected before extraction
started: configuration validation checked the backend name against the built-in list only, which
made every custom plugin OCR backend unusable once validation was wired intoextractand
extract_batch. -
Fixed the native C FFI library shipping without eleven features the crate advertises, so the
Java, Go, C#, Swift, Zig, and C bindings had no summarization, translation, analysis, HEIC,
captioning, ML redaction, or static-embedding support. The desktop dependency hand-maintained a
feature list that had drifted fromfull; a regression test now fails on any future omission. -
Fixed HTML pages fetched over HTTP(S) losing every format-specific metadata field: results were
reported astext/htmlwhilemetadata.formatstayed empty, because the extraction ran over the
crawler's pre-rendered Markdown and never reached the HTML extractor. Title, headings, Open Graph,
Twitter card, links, and structured data are now recovered from the page HTML. -
Fixed
pdf_options.hierarchy.enabledsilently producing no hierarchy: headings were detected and
then discarded unless the caller also set the unrelatedpages.extract_pages. Requesting the
heading hierarchy now enables the per-page tracking it requires. -
Fixed the bundled Tesseract build failing to configure on Windows when the MSVC developer
environment is not present, which broke building Xberg from source with the default OCR features. -
Fixed URL extraction reporting internally converted HTML pages as
text/markdown; results now
retain a validated, canonical source MIME type. -
Fixed
clear_post_processorsstopping at the first failed shutdown hook and permanently removing
enabled built-ins; it now attempts every shutdown, returns the first error, and restores built-ins
before the next post-processed extraction while custom processors remain cleared. -
Fixed VLM concurrency limits increasing concurrent local OCR work and raster memory use (#1465).
-
Fixed structured extraction forcing every caller schema to JSON Schema Draft 2020-12; validation
now honors the schema's declared draft while keeping external reference resolution offline
(#1539). -
Fixed hybrid PDF OCR dropping surrounding prose when a table-bearing bare-text page was
restructured alongside geometry-backed pages. -
Fixed automatic PDF OCR fallback reporting an empty success when OCR failed and no native text
remained; recoverable failures still return available native text with a warning. -
Fixed degraded VLM fallback output replacing denser OCR text, while abstaining from the density
comparison for short text and non-space-delimited CJK or kana content. -
Fixed Windows source and Ruby package builds failing on stable Rust while validating the
identity of staged Tesseract source directories. -
Fixed GCC 12+ WordPerfect builds by adding the standard header that declares
size_tbefore
compiling the pinned libwpd source. -
Fixed Ruby source-package installation by aligning the Gemfile and lockfile with the gemspec's
supportedrb_sysrange. -
Fixed generated Ruby development commands so Bundler and its tools use the active Ruby
interpreter, avoiding native-extension ABI conflicts on systems with multiple Ruby versions. -
Fixed generated Python optional constructor arguments so Pyrefly receives precise keyword types
without unused helper declarations. -
Fixed generated Dart tests for nested tagged unions, nullable payloads, and Flutter Rust Bridge
tuple accessors; added e2e analyzer coverage and refreshed the Dart lock file to the generated
Flutter Rust Bridge version. -
Fixed compressed image inputs with oversized declared dimensions exhausting memory during OCR,
layout and QR detection, image classification, re-encoding, HEIF conversion, or structured-image
rasterization; decoded allocations now obeysecurity_limits.max_content_sizeand are rejected
from the image header before pixel decode. -
Fixed PDF OCR fallback being suppressed for image-only pages when dot leaders or other
non-textual native content pushed the document below the alphanumeric-ratio threshold. -
Fixed process-global native PDF font-cache collisions that made glyph spacing, geometry, and
batch output depend on document order and concurrency when fonts used indirect width tables. -
Fixed Markdown OCR metadata so word counts and confidence statistics describe only text retained
after dictionary filtering; fully filtered output now reports zero words and omits confidence
quantiles. -
Fixed repeated bold PDF presenter labels and same-row legend keys being promoted to headings,
which could invert document hierarchy and fragment retrieval chunks. -
Fixed PDF OCR so fragmented, low-confidence, and dictionary-suspect non-empty text is retained
with a processing warning by default instead of silently emptying pages. Set
ocr.quality_thresholds.discard_suspected_ocr_noise = true(or the equivalent pipeline quality
threshold) to opt into the previous destructive filtering behavior. -
Fixed runtime crashes in system-linked Tesseract OCR builds by linking the required native exception-safety
shim. -
Fixed
xberg batchso mixed-success runs emit every successful document and every attributed
per-input error before returning a nonzero status; JSON and TOON timing slots remain aligned with
inputs, and TOON now uses the documented batch envelope. -
Fixed
xberg extract --ocr falseso it authoritatively disables implicit OCR fallback, overrides
conflicting loaded OCR routing, and rejects contradictory OCR flags. -
Fixed Tesseract preprocessing so deskew, denoise, contrast enhancement, and Otsu, adaptive, and
Sauvola binarization settings transform the OCR raster on native and WebAssembly backends;
none(withoffas an alias) preserves unthresholded grayscale when deskew is disabled,
sparse receipt-image fallback and faint colored text no longer lose content to global
thresholding, dark labels over bright map fills still receive Otsu preprocessing without
isolated or clustered dark artifacts triggering it, and
WebAssembly Tesseract now rejects images exceeding 4096 × 4096 pixels before decoding. -
Fixed OCR measurement tooling so line-filter comparisons score the intended ground-truth lines
and report filtering regressions accurately. -
Fixed the OpenAPI document's dangling Djot attribute reference so schema validators and client generators can
resolve every advertised component (#1505). -
XML and JSON content with unsupported specialized extensions now routes through the supported generic extractor
instead of failing MIME validation (#1507). -
File extraction now falls back to bounded content sniffing when a path has an unknown or missing extension (#1506).
-
Explicit
application/octet-streamhints now trigger configured MIME detection instead of being
treated as an authoritative document type. -
Fixed documentation-snippet fixtures that named non-existent result fields, which made the generated
snippets silently drop the affected presentation block: elementcontentis nowtext, tablerows
is nowcells, and the result pathskeywords,structured_data, anddocument_structureare now
extracted_keywords,structured_output, anddocument. -
Fixed EPUB extraction for
text/htmlspine items, named entities, declared non-UTF-8 encodings,
navigation documents, SVG fallbacks, nested tables, MathML, headings, images, and malformed HTML
(#1486, #1488-#1494). -
EPUB extraction now preserves usable chapters when another spine item fails and reports per-item
warnings instead of failing the whole document (#1491). -
Fixed EPUB metadata, EPUB 2/3 cover selection, DRM detection, and font-obfuscation handling (#1492,
#1494). -
Fixed PDF OCR and rendering for highly compressed scans, CCITT images, CFF fonts, maximum-size font
tables, malformed embedded fonts, rotated content, missing glyph warnings, and concurrent Pdfium
extraction. -
Fixed native PDF tracing so corrupt optional content is reported as a recoverable warning, while
mandatory cross-reference failures emit a single operation-boundary error without changing the returned error type. -
Fixed annotation-only PDFs so visible FreeText content is recovered into page-aware document text,
including when OCR replaces the page text, without exposing hidden, transparent, cropped, or
disabled annotations when annotation extraction is off. -
Fixed the Swift package manifest so SwiftPM no longer warns about a nonexistent target-relative license file.
-
Fixed scanned PDF extraction so CCITT parameters align with their filter in multi-filter streams,
referenced JBIG2 image masks are available to OCR, and stencil-mask polarity renders text as opaque. -
Fixed PDF reading order for dense two-column layouts, hanging clause numbers, split list markers,
and modest font-size changes on one baseline. -
Fixed PDF heading recovery for repeated bold section titles set at body font size while retaining
short bold labels, presenter attributions, and calendar legends as body text (#1513). -
Fixed PDF table extraction so multi-word cells, rule-less prose regions, OCR-derived tables, and
page-local table failures are handled correctly (#688, #1358, #1542). -
Fixed PDF Markdown and Djot output so native text is retained when structured conversion is
incomplete. -
Fixed PDF configuration so metadata suppression and header/footer settings are honored by every
backend; invalid or unsupported PDF and OCR settings now return configuration errors. -
Fixed OCR-backed PDFs so filtering, confidence thresholds, hierarchy, tables, formulas, lists,
bounding boxes, page boundaries, and partial page results are preserved consistently across output
formats and OCR backends (#1444). -
Fixed Tesseract caching, configuration, preprocessing, page segmentation, and font-size extraction.
-
Tesseract Markdown extraction now reports a
ProcessingWarningwhen dictionary filtering removes
physical text lines, including the number removed. -
OCR element hierarchy output now honors
build_hierarchyand contains only resolvable parent
references. -
Fixed Sceptre and PaddleOCR line grouping, region ordering, per-page resizing, table validation,
and font-size reporting. -
Fixed DOCX extraction for nested tables, VML images, text boxes, comments, fields, headings,
hyperlinks, headers, footers, table-of-contents entries, nested lists, and page attribution;
element output now preserves explicit page breaks and single-page documents report page metadata
consistently (#1452, #1460, #1503). -
Fixed PPTX extraction for malformed relationships, nested image paths, equations, fallback shapes,
comments, metadata, and security limits. -
Fixed spreadsheet extraction for hyperlinks, formulas, names, comments, hidden state, dates, and
OpenDocument metadata. -
Fixed ODT, ODP, iWork, HWP, DBF, RTF, email, and PST extraction across nested content, metadata,
binary data, folder traversal, and repeated text. -
Fixed Markdown, MDX, RST, HTML, DocBook, JATS, FictionBook, Djot, Org, YAML frontmatter, and Jupyter
extraction so supported structure and content are retained. -
Fixed
result.elementsso headings report their level (metadata.additional["heading_level"])
instead of every##-######heading collapsing into indistinguishableHeadingelements with
empty metadata;result.document.nodesalready carried the level correctly (#1504). -
Fixed CSV parsing for stray quotes and archive extraction order.
-
Fixed MIME routing so HTML is detected before the generic XML fallback and supported-format lists
reflect the active extractor registry. -
Fixed post-processing, chunking, enrichment, translation, NER, QR codes, captions, and caching so
extracted structure is preserved consistently. -
Fixed chunking presets so standalone and pipeline APIs apply the documented size and overlap while
preserving unrelated chunking settings. -
Fixed extraction timeout handling so timed-out work is cancelled.
-
Fixed configuration merging so changing one CLI option no longer erases sibling settings.
-
Fixed multipart API extraction to accept
jsonanddoctagsvalues foroutput_format. -
Fixed cache keys to reflect only settings that affect the corresponding extraction or OCR result.
-
Fixed model caching so OCR, embedding, and reranking settings no longer reuse incompatible models.
-
Fixed Node.js native-library loading, Swift iOS resolution, Windows DirectML packaging, and
cargo install xberg-cli(#1456). -
Fixed Docker image builds and reduced the CLI image to runtime dependencies.
-
Fixed API and packaging defects in the Python, PHP, Dart, Go, Java, C#, Kotlin, Elixir, Ruby, Zig,
and C packages. -
Fixed Windows wheel and gem packaging, manylinux compatibility, musl smoke tests, and dynamic
Tesseract builds (#1495, #1497). -
Fixed archive and ZIP validation for small compressed entries and impossible declared sizes
(#1496). -
Fixed batch extraction so configured caches are used and progress callbacks report completed items.
-
Fixed extraction configuration validation so invalid nested values, including OCR quality and
scanned-page thresholds, are rejected consistently by every public entry point. -
Fixed error classification so callers can distinguish all documented extraction failure categories.
-
Built-in path and byte extraction now always reports a recognized
extraction_method; custom
extractors retain explicit provenance and otherwise leave it unspecified. -
Fixed owned document classification so detected labels are written back to the returned document.
-
Fixed
ContentFilterConfig.include_watermarksso enabling it retains watermark content. -
Fixed
JsonExtractionConfig.flatten_nested_objectsso disabling it preserves nested objects instead of
flattening them. -
Fixed standalone-image and OCR-backed PDF results so preprocessing scale, dimensions, and DPI are retained.
-
Fixed Candle OCR configuration so supported backend options are validated and applied.
-
Fixed PaddleOCR-VL so the task selected when constructing the backend is honored unless a request
explicitly overrides it. -
Fixed keyword extraction so invalid n-gram ranges return an error instead of silently producing
empty results. -
Fixed builds that enable only the
apiormcpfeature. -
Fixed the
excel-wasmfeature so spreadsheet extraction builds for WebAssembly. -
Fixed WebAssembly configuration so unsupported managed credential providers are rejected explicitly.
-
Fixed the Swift package failing to link on Linux.
Package.swiftlinkedlibxberg_ffi.aalongside
libxberg_swift.a, but the Swift static library already folds the entire compiledxberg-fficrate
in, so every Rust core, std, and alloc symbol existed twice and the linker reported hundreds of
duplicate symbols. It also never asked for ONNX Runtime, leavingOrtGetApiBaseundefined. -
Fixed the public
clear_ocr_backends()andclear_renderers()leaving their process-global
registries permanently empty. Afterclear_ocr_backends()every later extraction failed with "No
available OCR backends"; afterclear_renderers()theCustomoutput-format path silently
downgraded DOT renders to plain text for the life of the process. Both now re-seed the built-ins
non-destructively, keeping user-registered entries. -
Fixed nested lists rendering as flat, blank-line-separated bullets in
pages[N].content: container
list markers are never page-tagged, so a page subset dropped them and every item was rewrapped in
its own single-item list. Also fixed figure alt text being dropped whenever a caption was present,
the VLM OCR probe reporting availability without checking credentials, and the PDF margin filter
judging rotated text runs by baseline origin. -
Fixed HEIC-enabled builds requiring a libheif newer than current stable distributions ship. The
prebuilt artifacts link libheif dynamically and were built against 1.21 APIs, so the PHP extension
failed to load on Debian 13 withundefined symbol: heif_image_get_plane_readonly2. The floor is now
1.19, with version-gated fallbacks (#1541). -
Fixed PDF text collapsing on itself when a font's
/Widthsarray declares 0 for an ordinary glyph.
Extraction now falls back to the embedded font's own advance for such codes, while an explicitTJ
displacement stays authoritative and genuine zero-width combining marks remain overlays. -
Fixed automatic PDF OCR replacing a page's native text with a substantially poorer recognition. OCR
output for a page whose native text was independently judged healthy is now rejected when it retains
under half that page's alphanumeric characters. -
Fixed OCR of a single detached page image being attributed to page 1. Local image indices were used
as document page numbers, so warnings named the wrong page and the rejected-page filter discarded
OCR elements, tables, and formulas belonging to a different page than the one rejected. -
Fixed XML extraction narrowing element depth to
u8before clamping, so an element nested more than
255 levels deep wrapped to a low heading level in release builds and panicked in debug builds, before
the configuredmax_xml_depthlimit ever applied
(#1474). -
Fixed PDF XMP metadata losing text fragments split around an entity boundary: named and numeric XML
references in XMP scalar and sequence values are preserved instead of the surrounding text being
truncated (#1475). -
Fixed image-level OCR running again over a page-sized PDF XObject on a page whose native text had
already been extracted, which duplicated the page's content and paid for a second OCR pass
(#1479). -
Fixed the musl (Alpine) native artifacts failing to load. The published Java, C#, Zig, C, and Elixir
artifacts shipped without ONNX Runtime's transitive closure — libprotobuf-lite, thelibabsl_*set,
libre2, and libicu. Both musl images now vendor the fulllddclosure and hard-fail the build if
anything is unresolved. A host runtime that links libstdc++ itself still needs libstdc++ 15 or newer
in the process, because a bundled copy cannot win once the soname is already mapped. -
Fixed a DOCX or PPTX relationship targeting
../media/image1.png— the ordinary OPC shape for an
image at the package root — being rejected by the traversal check and dropped, so the image went
missing from extraction. Container-relative names now resolve boundary-relative. -
Fixed OCR of rendered PDF pages assuming a 72 DPI raster when pages render at 150 DPI, so DPI
normalisation computed a 2.48x upscale, hit the dimension clamp, and reported a resolution hint of
179 for what was really a 372 DPI image. Also fixed image DPI normalisation being skipped entirely in
candle-backend and VLM-only builds. -
Fixed layout detection marking real figure and drawing text as page furniture, which the renderer
then discarded, so labels such asSITE PLANandLEGENDdisappeared from scanned documents. A
Picturehint now means a figure was detected, not that the text is decoration, and furniture hints
only match short text. -
Fixed the Docling-compatible endpoint discarding OpenWebUI's extraction parameters. OpenWebUI sends
one form field per key rather than a JSON blob, so settings made in its admin UI produced identical
output with or without them (#1462). -
Fixed CLI flags being silently discarded.
--ocr-backend,--ocr-language,--ocr-auto-rotate, and
--ocr-backend-optionswere dropped unless--ocr truewas also passed, so--ocr-scanned-pages --ocr-backend sceptreran Tesseract with no error;--ocr-scanned-pagesalone returned an empty
document at exit status 0; and--chunk-sizewas a no-op without--chunk true. -
Fixed legacy
.docextraction emitting every field's instruction — its URL, switches, and
screen-tips — verbatim as prose, and the non-breaking hyphen being dropped with the other control
characters, fusingtwenty-oneintotwentyone. -
Fixed paragraph grouping only breaking when a line starts a numbered section and never when the
previous line was one, so a subsection heading followed by unnumbered lines at the same size and
weight was merged into the following prose
(#1467). Consecutive numbered headings are likewise
no longer welded into a single paragraph
(#1386). -
Fixed the PDF pipeline stripping a list item's printed marker and discarding it, leaving renderers to
synthesize a position, so a document whose clauses are cross-referenced by their printed label was
renumbered —B.rendering as1.and(a)as1.. -
Fixed
candle-trocraccepting a whole page and returning invented text. TrOCR is trained on single
cropped lines and force-resizes any input, so a multi-page document exited successfully with text
appearing nowhere in it. Input taller than a plausible line crop is now rejected. -
Fixed inline
<svg>elements being discarded during HTML extraction even withextract_images
enabled (#745). -
Fixed an explicitly requested GPU execution provider silently running on CPU.
is_available()
reports only compile-time support and ORT's session builder defaults to not erroring on failure, so
an explicit CUDA, TensorRT, or CoreML request that failed to load was swallowed. Explicit requests
now fail;Autokeeps its silent fallback. -
Fixed DOCX documents with legacy VML picture markup being rejected as
NestingTooDeep, and the
inverse hole where content inside drawings, table property helpers, the table grid, and streaming
section properties was never measured against the depth cap at all. A flat 600-row table of real
depth 8 previously leaked over a thousand levels and was rejected outright
(#1395). -
Fixed
XBERG_LLM_API_KEYandXBERG_LLM_BASE_URLfabricating a structured-extraction config with an
empty model and schema, so any deployment that merely had an LLM key in its environment ran the
post-processor on every document and failed every one
(#1421). -
Fixed two PDF paths aborting or failing the whole request: a
/ModDatewhose raw bytes decode to a
replacement character sliced astroff a char boundary and panicked, which across the Go FFI
boundary aborts the process before anycatch_unwindframe is consulted; and a rasterizer panic on a
page with damaged content streams unwound through the async boundary and lost every other page's text
(#1422, #1408). -
Fixed keyword extraction panicking on a language hint whose first character is multi-byte.
-
Fixed legacy
.pptslide numbering and image extraction. Slide numbers were the ordinal of a text
block in a joined string, so a trailing paragraph mark cut one slide into several; they now come from
the slide containers in persist order. The OLE/Picturesstream was never opened, so.ppt
extraction never produced an image (#1418,
#1417). -
Fixed PPTX slides without a title losing their page number
(#1413). -
Fixed URL extraction reporting no crawled URLs, because the result field is no longer populated
upstream. The URLs are now derived from the crawled pages, deduped in first-seen order. -
Fixed PDF page-number stripping deleting real table data. The decision was made from one paragraph's
text, so any short numeric cell matched; it now requires a margin band, a stable horizontal slot
across pages, and a progressive sequence to agree
(#1411). -
Fixed PDF paragraph breaks never being detected on a normally-set page, so a whole memo — date,
salutation, body, sign-off — came back as one line. The vertical advance is now compared against the
body leading, which is scale-free. -
Fixed detected PDF tables being injected on top of native text that already contained them, so the
same content was rendered twice. -
Fixed non-HTML raw blocks being written verbatim into styled HTML output. ODP speaker notes and
master-page text, Org source, script and style bodies, and Djot raw blocks all reached the page
unescaped, so any<in them corrupted the document structure. -
Fixed the PyPI
xberg-cliwheels shipping without their native libraries. The build hook
force-included siblings with a macOS-only glob, so every Linux shared object staged beside the binary
was dropped, and the musl wheel shipped only the launcher script. An incomplete platform payload now
fails the build instead of publishing a wheel that installs and cannot run. -
Fixed OCR'd PDF pages reporting bounding boxes in raster pixels while digital pages report PDF
points, with nothing in the response distinguishing the two spaces. Node, hierarchy block, chunk page
span, and table bounding boxes are now converted to page points with a bottom-left origin
(#1423). -
Fixed OCR on pages carrying a
/Rotateentry. Backends now declare how they cope with a rotated
raster, so a backend that requires an upright page is handed one with its geometry mapped back, and
PaddleOCR receives the page rotation as a sort key. Auto-rotation composes with the page hint instead
of double-correcting it. -
Fixed PDF text and tables on rotated pages. Rotated-text repair reconstructs the reading frame but
only when rotated spans are at least 20% of a page's characters, so a single rotated caption no
longer costs the upright majority of the page its whitespace structure, and heuristic table
reconstruction clusters cells on the table's own axes rather than raw page space
(#1358). -
Fixed the OpenAPI document omitting types that client generators need: second-order nested component
schemas are now registered, along with the PDF, office, and transcription schema groups and the415
and429responses the extraction endpoints can return
(#1424). -
Fixed
code_intelligencebeing hardcoded toNone, so the documented metrics, imports and exports,
comments, docstrings, symbols, and diagnostics never reached callers
(#259). -
Fixed Whisper timestamp tokens leaking into transcripts as literal text. They are not marked special
in the tokenizer vocabulary, so they survived decoding; they are now paired into segments, emitting
one paragraph per segment with start and end times. -
Fixed
cargo add xberg --features fullfailing to link on Windows MSVC, where a transitive build
script forces/MTwhile Rust defaults to/MD, killing the build withLNK2038
(#1389). -
Fixed
show_download_progresshaving no readers anywhere on the embedding, sparse-embedding,
reranker, and late-interaction model configs, so the documented option did nothing. -
Fixed
split_and_extractrebuilding each segment from a handful of fields, dropping keywords,
entities, summaries, chunks, warnings, and the rest of the enrichment that extraction produced, and
an off-by-one in the chunk image-index remap that pointed chunks at the wrong image. -
Fixed
target_dpi,max_image_dimension,auto_adjust_dpi,min_dpi, andmax_dpihaving no
readers: every preprocessing config was built with defaults, so these settings were dropped
(#209). -
Fixed declared telemetry that never emitted. The cache-hit, cache-miss, and batch instruments were
declared but never recorded, and the pipeline and batch operations, five of the eight pipeline stage
spans, and the extractor-priority and batch attributes were likewise never recorded, so filtering on
them returned nothing (#332,
#282). -
Fixed an injected cache backend never being consulted and
ProgressSink::emithaving no caller on
single extraction;extract_batchwas already correct. A bytes-input cache hit now short-circuits
extraction and coarse start, complete, error, and cache-hit events are emitted. -
Fixed renderer output completeness: JSON silently dropped page breaks, footnote references and
definitions, citations, slides, definition terms, admonitions, raw blocks, and metadata blocks
through a catch-all arm; styled HTML opened a section for each slide that was never closed and never
rendered the slide title; and formulas rendered as preformatted code, which KaTeX and MathJax cannot
pick up, and are now delimited display math. -
Fixed footnote definitions never appearing in JSON output, and a definition present in the document
but never referenced being dropped from rendered output entirely
(#68). -
Fixed plugin-produced documents losing content at the bridge. The conversion into the internal
document droppeduris,children,annotations,processing_warnings,llm_usage,pages, and
ocr_elements; native renderers reached through the public entry point emitted an empty shell; and
pre_rendered_contentwas ignored for HTML and JSON output. -
Fixed CRLF documents collapsing into a single paragraph. Ten call sites split paragraphs on a bare
double newline without normalising line endings first, affecting email and PST bodies, OCR backend
output, plain text, and Djot conversion (#227). -
Fixed MIME aliases that were advertised as supported and then failed as
UnsupportedFormat, because
the registry looks up by exact string with no alias resolution.application/wordperfect,
application/x-quarto, and four audio and video transcription aliases now route to the same
extractor as their canonical type. -
Fixed three internal OCR plumbing keys being copied into user-visible document metadata.
Security
-
Bounded DOCX image and iWork archive member reads by the member's declared uncompressed size
instead of trusting that declaration. A crafted document could forge a small declared size in the
ZIP central directory while carrying a deflate stream that inflated to multiple gigabytes,
exhausting memory during DOCX image extraction (images.extract_images) or.pages/.numbers/
.keyextraction. Reported by Syed Anas Mohiuddin
(GHSA-85w9-wqcq-x48r). -
Pinned downloaded Tesseract, Leptonica, and English tessdata inputs to immutable revisions with
verified sizes and SHA-256 digests, race-safe content-addressed caches, private build directories,
and bounded fail-closed archive extraction. -
Structured extraction now resolves caller-provided JSON Schemas strictly offline and rejects
external HTTP and file references without performing I/O. -
REST and MCP requests can no longer override LLM credentials, provider registrations, or other
server-controlled settings. -
Hardened ZIP accounting against overflow, impossible sizes, and compression-ratio bypasses.
-
Hardened DOCX, PPTX, and EPUB relationship resolution against container traversal, malformed UTF-8,
NUL bytes, drive-letter paths, UNC paths, and symlink escapes. -
Added bounded EPUB traversal and retained-content accounting to prevent resource-limit bypasses.
-
Cache namespaces are validated before directories are created.
-
Redaction now reports only content that was actually removed, never exposes pre-redaction element
text, and rejects invalid strategies instead of silently falling back to masking. -
Hardened the native PDF engine against crafted documents that abort or hang the host process. A
self-referencing/Names /EmbeddedFilestree and deeply nested array or dictionary brackets each
recursed until the stack overflowed, which is an abort nocatch_unwindcan contain; a negative
/Welement in an xref stream, a reversedbfrange, a non-hexToUnicodedestination, an all-NaN
font-size set, and unchecked/Widthx/Height,/N, and/VerticesPerRowproducts each panicked
or allocated without bound; anddecode_stream_with_params, the entry point every production call
site uses, applied no ratio or size guard at all. All were reachable fromextract_bytesunder
default configuration. -
Bounded every ZIP, TAR, and 7z member read against
SecurityLimitsrather than against the size the
archive declares for itself, since a declared uncompressed size is not a bound and the aggregate
check previously ran only after the member was fully resident. Covers generic archives, ODT, ODP,
EPUB, HWPX, PPTX, XLSX, and OOXML embedded objects, and adds the compression-ratio and aggregate-size
validation that PPTX, XLSX, and DOCX were missing. A nested ZIP no longer overflows the stack. -
Clamped or rejected document-declared counts that reached an allocation or a slice unchecked: HWP
table row and column counts, HTML and EPUBcolspan/rowspan, DOCXw:ilvl,w:gridSpan, and
w:outlineLvl, PPTXa:pPr lvl, RST simple-table column ranges, JATSdate-type, EPUB link-label
offsets, PPTX relationship targets, and the hOCR parser's and annotated-text renderer's byte-offset
slices. Each was an out-of-bounds or char-boundary panic, or an allocation abort, on ordinary
untrusted input. -
security_limits.max_files_in_archiveis now enforced by every OOXML container. XLSX never checked
it, DOCX enforced a hardcoded 10,000-entry cap instead of the configured one, PPTX had no entry check
at all, and embedded-object extraction walked embeddings uncapped
(#1449). -
EPUB packaging XML now counts real OPF nesting depth against the configured limit and accepts legacy
DTD declarations without resolving external or amplified entities, so a crafted package can neither
bypass the depth budget nor pull in outside content
(#1477, #1478). -
Native PDF tracing no longer carries document content. Decoded page text was emitted verbatim at
TRACE, embedded font names appeared in trace events and in the glyph-dropProcessingWarning
message, and parser, xref, and recovery failures were logged by formatting the underlying error
string. Failure paths now emit a structurederror_codewith an optional byteerror_offset, and
font names are redacted in the warning text. -
Bounded the native PDF reader's internal caches so a malformed or hostile document cannot grow them
without limit: the object-stream cache evicts to a byte budget and rejects oversized entries, font
identity hashing stops at a byte budget and a reference-depth cap (both recorded in the hash so
distinct fonts stay distinct), and the xref recovery-marker set is capped.