gateway v0.3.1

A point release of the attested-input gateway: three changes since v0.3.0. The document adapter now reads its deadline throughout opening and reconstructing a PDF. The version 1 attachment record admits a PDF the deadline stopped opening, and its encryption fields say what the reader established. The web adapter can discover a site's other pages.

The document adapter reads its deadline wherever it works

The adapter's deadline (25 seconds by default, --timeout) is meant to end a reading with an honest partial record — the pages completed and a timeout — rather than let the gateway's source timeout kill it with no record at all. Before this release, three parts of opening a PDF could run for as long as the file made them without reading that deadline. Each now reads it at a stated cadence, and the README's table states each figure:

• The scan that rebuilds a missing cross-reference (#158, closing #155). It matched object headers with one regexp pass over the whole file before its first deadline read. It now reads the deadline before it begins and between parts of 64 KiB, with every byte it inspects charged, so at most 65,538 inspections lie between two reads. It finds exactly the headers the regexp found, as a differential test and a fuzz target hold, and it is faster: about 6–8 ms on a 60,000-object file where the regexp took 94 ms. An expired deadline now costs no scanning at all.
• One object's parse (#160, closing #157). A document's lexer reads the deadline every 16 KiB it advances over, inside whitespace, comments and strings as well as between tokens: at most 20,480 bytes between two reads.
• Reconstruction's other loops (#160, closing #159). The trailer search, the index of stream ends, the key sort, the copy of a trailer's members and the object-stream header now read the deadline every 64 KiB searched or every 4,096 entries. The worst gaps measured before were about a second on a 1 GiB file; after, a few milliseconds.

A deadline read as passed now stops the document for good. Nothing read after it is cached or published, and the record says timeout. Where a deadline and another failure are both met, the one met first in reading order is reported.

The cost on ordinary files is small but measurable: extraction is 1–2% slower on real PDFs, measured pinned and in shuffled order against a control. The deadline cadence is per-token work.

A change to the version 1 attachment record (#160)

A document not opened is now exactly one of two things: reported pdf-encrypted, or stopped by the deadline while it was opened, before its encryption was established, with the timeout as its one error and no page counted. Before, attachment.Check, the reference checker of the record, refused the second shape. The gateway's routes that check records — Drive, Gmail, web, resource and connected-source — therefore turned a timed-out encrypted PDF into processing-failed. pdf-encrypted keeps its meaning: a password is required, or the handler or revision is not one the adapter implements.

The encryption fields state what the reader established. A failure of the encryption dictionary met before the deadline stands as pdf-encrypted. A reading that opened the document before the deadline records opened: true. A declaration the deadline left unread is null for that reason. If a rebuild of the cross-reference was stopped before completing its own reading, the record describes the dictionary as last read in full. docs/design/attachments.md states these rules at the encryption row and step 4. testdata/attachments/examples/failed-timeout-encrypted.json is an example of the new shape. Consumers that apply the old rule should admit it.

Website discovery (#156)

adapter-web --discover, advertised as web-discovery in the source catalog and the local plan, returns a signed navigation manifest of a public HTTPS site's other pages. The selected pages then go through ordinary web acquisitions, each with its own receipt. Discovery stays on the exact origin and checks robots.txt and every redirect. It caps pages, depth, links, bytes, requests and elapsed time. It takes no credentials and renders no JavaScript.

Review

#158 and #160 went through the gateway's cross-vendor review regime. #158 had two rounds; #160 had five, plus two same-vendor verification workflows. Every round and its disposition are recorded on the pull requests. Gaps of the same kind in decoding, decryption and the page walk's cycle shortcut, which predate this release, are filed as #161.

The frozen corpus is unchanged: 30 canon vectors and 41 store vectors, and gateway conform reports 0 disagreements on the tagged source.