Skip to content

v0.6.0 — salvage, kept out of the field callers paint from

Choose a tag to compare

@tannevaled tannevaled released this 27 Aug 13:43
· 25 commits to main since this release
355e5b2

Three defects were put to this release. One of them did not exist, and that is reported as plainly as the two that did.

A broken filter chain kept what it had, and now says it is salvage

Half of this was already here and lying: flateDecode returned a damaged stream's inflated prefix with a nil error, so 37 streams — 33 in 18 real forms, 4 in fixtures — were reported as clean decodes of a fragment. The other half was missing: 263 streams in 212 of the 1 633 real forms (13.0%) failed and yielded nothing at all.

The recovered bytes are now kept structurally apart from the clean ones. Decoded.Data holds only what a filter decoded; Decoded.Undecoded holds what the chain could not get past, set instead of Data, never beside it. Writing that guarantee as a test found two live holes: cryptFilter was returning ciphertext as Data, and v0.5.0's own CCITT change meant a fax with a bad /Columns failed inside the chain with its encoded bytes landing in Data — a fax handed to a stencil path as if it were samples, which is exactly the shape of the defect render v0.10.0 had just fixed. The guarantee is a type, not a flag.

/Length against the cross-reference table — there was nothing to fix

The cross-check is already in parse.go. Measured over 344 367 streams in 4 283 files: 47 streams need the scan fallback, 0 are ambiguous, 0 truncate, 0 overrun the next object header. No change was made, and none was needed.

A masked error, worse than described

Two faults. The masking bites 30 of 30 arXiv open failures: every one reported no startxref in the last 4096 bytes where the rebuild had actually found the file holds no indirect objects (28) or no document catalogue found (2). And an ordering fault — repair() read object streams before the decryption key existed, so an encrypted file with broken tables was rebuilt from unreadable bytes. That one bites 0 files in either corpus; it ships anyway, with the negative result stated rather than dressed up.

Three the search turned up on its own

/Crypt is the biggest real gap: 209 of the 263 failures, in 209 files — 12.8% of the real forms. Implementing only the filter would be measurably the worst state (209 of 209 decode "cleanly", 0 of 209 give readable XMP); both halves together give 209 of 209 readable.

A second map-order non-determinism, in indexObjectStreams — distinct from the Expanded fix in v0.4.2, and still giving 5 answers in 5 runs with that fix in. The tie-break now matches what the header scan already does: later in the file wins.

/StmF and /StrF both /Identity means nothing in the file is encrypted and no password is needed. 3 of 495 encrypted files; all three now open with 30, 1 and 30 pages — poppler's exact numbers.

Measured

118 843 arXiv files pages 138 333 → 138 333; content bytes 30 423 755 330 → 30 423 755 328; 0 panics
1 633 real forms every per-file content hash identical; strict refusals 263 → 87
1 184 files cut to 92% pages returning nothing 591 → 0; content 126 466 671 → 137 214 633 (+8.50%), of which 137 214 607 bytes are clean decodes and 26 bytes are flagged salvage; determinism 6-runs-5-answers → 3-runs-1-answer
cost flat on healthy files (563 MB allocated either side); +12% allocation on the damaged path, wall clock inside noise

The two arXiv files differing by one byte differ by a redundant newline for an empty stream in a /Contents array; operation count and operator-sequence hash are identical on both.

100% statement coverage, go vet and -race clean, nine cross-compile targets — each built one per line, with bogus GOOS/GOARCH as controls. Two fuzz targets added where there were none. Nothing outside the standard library.