Skip to content

Add default safe snapshot processing - #6

Merged
magiccodingman merged 43 commits into
mainfrom
agent/default-snapshot-processing
Jul 27, 2026
Merged

Add default safe snapshot processing#6
magiccodingman merged 43 commits into
mainfrom
agent/default-snapshot-processing

Conversation

@magiccodingman

@magiccodingman magiccodingman commented Jul 26, 2026

Copy link
Copy Markdown
Owner

What changed

  • add a separate Snapshot.Processing library built on the protocol processor contract
  • enable safe processing by default in the CLI build path
  • enforce one absolute HTTP/HTTPS canonical whose normalized path matches the captured route by default
  • compact generated snapshot HTML, ordinary safe-to-remove comments, inline JSON/JSON-LD, and inline <style> content
  • deliberately preserve all inline JavaScript and disable NUglify's JavaScript processing
  • validate transformed output and fall back to the unminified snapshot whenever structural or executable-content invariants change
  • add archive-to-archive processing that rewrites only manifest entries marked Snapshot
  • preserve source index.html, aliases, prefix gateways, assets, and hosting artifacts unchanged
  • encode generated HTML directly into ZIP entry streams with pooled buffers, avoiding full UTF-8 byte-array copies
  • serialize the manifest directly into its ZIP entry stream
  • enumerate source files directly instead of retaining a second sorted filename array
  • report processing failures on the exact affected route and reject the final artifact

Default behavior

snapshot build now performs processing automatically. Canonical mismatch and invalid inline JSON are route failures and build errors by default.

Simple escape hatches remain available:

  • --no-minify keeps validation and preserves each snapshot HTML string exactly
  • --no-processing disables the complete layer
  • --canonical-policy warning|off relaxes canonical enforcement
  • granular switches can preserve HTML comments or disable HTML, inline JSON, or inline CSS minification

JavaScript is never minified, renamed, bundled, tree-shaken, or rewritten by this processing layer.

Architecture

Snapshot.Protocol owns only ISnapshotProcessor and invokes it as successful render results flow into the archive. Snapshot.Processing owns the concrete DOM parsing, validation, conservative minification, semantic verification, and standalone archive processing implementation. Playwright remains only an executor.

This keeps the processor reusable by future executors, including the planned Blazor WebAssembly/browser implementation, without depending on SQLite, Playwright, or a desktop-only storage model. The normal CLI remains a one-pass build; existing archives can use the same processor through snapshot process.

Scope and isolation

Processing applies only to generated route snapshots, including /index/index.html for the rendered root route. It never processes the original /index.html loader, case aliases, prefix gateways, source assets, or hosting artifacts.

The standalone archive processor streams unchanged entries into a new ZIP, rewrites only manifested Snapshot entries, regenerates lengths and hashes, rewrites the manifest, validates the destination, and publishes it atomically.

Memory behavior

Rendered page bodies and ZIP bytes do not accumulate across the build. Each route is processed as it arrives, encoded directly into its ZIP entry, and released. Memory therefore scales primarily with browser concurrency and the largest simultaneously rendered pages rather than total site HTML size.

Route plans, route results, and manifest records still scale linearly with the number of real output entries; they do not retain entry contents. There is no separate large-site pipeline or scale flag.

Validation

GitHub Actions CI run 87 is fully green:

  • Ubuntu, Windows, and macOS builds and tests passed
  • processing unit and archive-isolation tests passed
  • canonical processing errors are verified to mark their route failed and prevent artifact publication
  • --no-minify validation-only behavior preserves the exact HTML string
  • package creation, compact CLI validation, and local packed-tool installation passed
  • real Chromium/Blazor end-to-end builds passed with standard processing enabled
  • the original root loader remains byte-for-byte unchanged in end-to-end and archive-processing tests

The PR remains draft for a final TruthGate build and artifact-size review.

@magiccodingman
magiccodingman marked this pull request as ready for review July 27, 2026 00:34
@magiccodingman
magiccodingman merged commit 74fb234 into main Jul 27, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant