Skip to content

v1.7.1 — content-stream conformance rules (§6.2.2-1/-2, §6.1.10-1, §6.1.13-10) - #131

Merged
Tim81 merged 9 commits into
mainfrom
v1.7-content-interp
Jun 21, 2026
Merged

v1.7.1 — content-stream conformance rules (§6.2.2-1/-2, §6.1.10-1, §6.1.13-10)#131
Tim81 merged 9 commits into
mainfrom
v1.7-content-interp

Conversation

@Tim81

@Tim81 Tim81 commented Jun 21, 2026

Copy link
Copy Markdown
Owner

v1.7.1 — content-stream interpreter cluster

Adds four PDF/A-2b/2u/2a preflight rules driven by an in-process content-stream scan, continuing the v1.7 — In-process preflight milestone (#9). All are scoped to page content streams and cross-validated against veraPDF 1.30.2.

Clause Rule Status
§6.2.2-1 Content-stream operators must be defined in ISO 32000-1 (even inside BX/EX) Partial
§6.2.2-2 A page referencing named resources needs an explicitly associated /Resources dict Partial
§6.1.10-1 Inline-image filter must be a permitted ISO 32000-1 Table 6 filter (not LZW/Crypt/JPX) Partial
§6.1.13-10 Composite-font embedded CMap must not produce a CID > 65535 Partial

Partial = page content streams only; form XObject, Type 3 glyph, and annotation appearance streams are deferred (under-scanning is false-positive-safe).

Parity coverage (build-verified via ConformanceCatalog.Coverage): PDF/A-2b 79.5→80.9%, 2u 79.8→81.2%, 2a 77.1→78.4%.

Quality

  • Each clause was built with an empirical veraPDF STEP-0 probe, then Opus-reviewed. Review caught and fixed two real over-rejections before merge:
    • the reader lexer emits true/false/null as Keyword → an inline-image /I true was nearly flagged as an unknown operator (§6.2.2-1) / filter (§6.1.10-1);
    • §6.1.13-10's code-splitting is now codespace-aware (was a fixed-width guess that could synthesize a wide CID).
  • A post-batch adversarial false-positive sweep caught a third FP — §6.2.2-2 flagging /DeviceRGB etc. on a resource-less page — now fixed (7f7d37d), as device/Pattern colour names resolve without a resource lookup.
  • 513 conformance tests (399 in-process + 114 veraPDF oracle), format clean.

Docs

Tim81 and others added 9 commits June 21, 2026 17:52
…6.2.2-1)

A new ContentStreamOperatorRule tokenises each page content stream with the
reader's lexer and flags any operator keyword not in the ISO 32000-1:2008
Annex A (Table A.1) set — including operators bracketed by BX/EX, which per the
clause do NOT exempt them. veraPDF 1.30.2 confirmed the direction (an unknown
'zz' fails 6.2.2-1, and still fails inside BX..EX; clean writer output passes).
Three veraPDF-cross-validated oracle fixtures plus in-process unit tests cover
unknown / BX-EX / clean / empty / report-once.

Opus review fixed a false positive: the reader lexer emits the value keywords
true/false/null as Keyword tokens, so an inline-image `/Interpolate true` or a
BDC inline-dict boolean/null would have been flagged as an unknown operator —
which veraPDF does not do. They are operands and are now skipped; a regression
test (verified to fail without the fix) guards it.

Marked Partial, not Implemented: only page content streams are walked. Form
XObject, Type 3 CharProc, and annotation appearance streams — which veraPDF also
scans — are deferred; under-scanning is false-positive-safe.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RghjbtTZ9hn9P8A1JEgXVq
The package ships to NuGet (embeddable Liberation substitutes for the
standard-14 fonts, used via doc.EmbedStandard14Font to get PDF/A-conformant
standard-14 text) but was missing from the packages table.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RghjbtTZ9hn9P8A1JEgXVq
Surface the v1.7 flagship preflight package alongside the Kernel badge.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RghjbtTZ9hn9P8A1JEgXVq
…partial (§6.1.10-1)

A new InlineImageFilterRule parses each inline-image dictionary (BI..ID) in a
page content stream and validates the /F (or /Filter) value — single name or
array — against the inline-image-permitted subset of ISO 32000-1 Table 6:
ASCIIHexDecode/AHx, ASCII85Decode/A85, FlateDecode/Fl, RunLengthDecode/RL,
CCITTFaxDecode/CCF, DCTDecode/DCT. LZWDecode/LZW, Crypt, JPXDecode and any
other name are flagged. veraPDF 1.30.2 confirmed: both /F and /Filter keys are
honoured, JPXDecode is rejected, and a bad array member fails.

The dict parser guards the true/false/null value keywords (the reader lexer
emits them as Keyword) so an inline-image `/IM true`/`/I true` is not mistaken
for a filter, and discards non-/F arrays/dicts (e.g. /DP DecodeParms) without
false-positiving. Binary sample data is skipped via the shared
ContentStreamUsage.SkipInlineImageData helper.

Marked Partial (same scope as §6.2.2-1): only page content streams are walked;
inline images inside Form XObject / Type 3 CharProc / appearance streams are
deferred. 13 in-process unit tests cover the flagged and permitted paths.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RghjbtTZ9hn9P8A1JEgXVq
… (§6.2.2-2)

A new InheritedResourceRule flags a page whose content stream references named
resources but whose page dictionary has no own /Resources key (so the names
resolve only through Pages-node inheritance). STEP-0 probing of veraPDF 1.30.2
pinned the exact trigger, which is narrower than the clause prose suggests: the
rule fires only when the page dict itself lacks a /Resources entry. Any
/Resources on the page — even an empty << >> or one missing the used sub-dict —
satisfies veraPDF's "explicitly associated" test, so those are not flagged
(confirmed across 5+ probes). page.Get(/Resources) reads the page's own entry
without walking /Parent, giving exactly that predicate; VellumPdf writer output
always emits /Resources, so it is never flagged.

ContentStreamUsage gains a PaintedShadings set (the `sh` operator); the rule
reports used Font/XObject/ExtGState/ColorSpace/Shading names. Pattern (scn/SCN)
and Properties (BDC/DP) names are not detected — they need stateful colour-space
/ marked-content tracking, and omitting them only under-detects (never
false-positives). Marked Partial with that note; page content streams only.

Seven in-process unit tests: three inherited-resource positives (font/xobject/
extgstate named in the finding) and four no-finding guards (explicit resources,
no content, empty page, writer-produced PDF/A).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RghjbtTZ9hn9P8A1JEgXVq
….13-10)

A new CidRangeRule checks Type 0 fonts whose /Encoding is an embedded CMap
stream. It parses the CMap's codespace, cidrange, and cidchar sections, walks
each page's content tracking the current font (Tf), and for every text-show
operator (Tj/TJ/'/") resolves the shown character codes to CIDs and flags any
CID > 65535. veraPDF 1.30.2 confirmed the semantic is render-time, not
declaration-time: it gates on CIDs actually produced by shown text (a font
selected but never used does not fire), and on the range END, so a cidrange
<0000><00FF> 65500 (max 65755) is flagged when 00FF is shown.

Identity-H/V are structurally bounded at 65535 and never checked; predefined
named CMaps are deferred (no Adobe character-collection table), as in
§6.2.11.3.1-1 — hence Partial.

Opus review hardened the code-splitting: the original draft split text into
codes with a greedy 2-byte-then-1-byte guess, which for a 1-byte/mixed
codespace could synthesize a wide code that maps past 65535 and over-reject. It
now decodes codes per the CMap's begincodespacerange (ISO 32000-1 §9.7.6.2), so
only valid codespace codes are looked up — matching veraPDF and removing the
false-positive vector. A regression test (1-byte codespace with an inconsistent
2-byte cidrange) locks it; nine in-process unit tests cover the firing and
no-finding paths. No oracle fixture — an embedded-CMap Type0 PDF trips unrelated
findings (same reason §6.2.11.3.1-1 added none).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RghjbtTZ9hn9P8A1JEgXVq
… resources

Adversarial sweep caught a false positive: a page with no own /Resources that
selects /DeviceGray, /DeviceRGB, /DeviceCMYK or /Pattern via cs/CS was reported
as referring to an undefined inherited resource. Those four names are resolved
directly by cs/CS without a /Resources /ColorSpace lookup (ISO 32000-1 §8.6.3,
Table 73; §8.6.8 for Pattern), so a resource-less page using only them is valid
PDF/A — veraPDF 1.30.2 accepts it. InheritedResourceRule now excludes the four
direct colour-space names before reporting. Regression test added (verified to
fail without the filter).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RghjbtTZ9hn9P8A1JEgXVq
- CHANGELOG: stamp the stranded [Unreleased] block as [1.7.0] - 2026-06-21 (the
  release tag missed the rollover), add the [1.7.1] content-stream-rules entry,
  and backfill the v1.5.3–v1.7.x comparison links.
- README: fix the roadmap (xref/object streams shipped in v1.7, not pending;
  tag Linearization v1.8 / Barcodes v1.9; add the native-AOT preflight CLI #130),
  and note content-stream + embedded-CMap CID coverage on the Conformance row.
- VersionPrefix → 1.7.1.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RghjbtTZ9hn9P8A1JEgXVq
Per milestone decision, the native-AOT vellum-preflight CLI is the capstone of
the v1.7 In-process preflight milestone (#9), shipped as a later v1.7.x once
coverage is high enough — reflected in the roadmap row.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RghjbtTZ9hn9P8A1JEgXVq
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant