Skip to content

Spec v3.3.0: Delta Encoding for the Generic Profile (§10a)

Choose a tag to compare

@blackwell-systems blackwell-systems released this 12 Jul 06:09

Delta Encoding for the Generic Profile (Section 10a)

New Section 10a extends delta encoding to the generic profile: a keyed row diff with ## added / ## changed / ## removed sections, keyed on a designated identity column (@id in the field declaration + required key= in the header). ## changed replaces a whole row by identity (no field-level patch in this version).

  • Reuses the graph delta mechanism unchanged: the three-outcome echo handshake (§10.3), the gcf-pack-root-v1 framework and unknown-algorithm fallback (§10.2), atomic delta application (§10.4), and session scope (§9.3). Generic-specific additions are the ## changed section and a row-based canonical pack-root (§10a.3).
  • Opt-in and bilateral: a delta is emitted only after the consumer echoes a pack_root the server recognizes; the decoder applies unchanged / delta / full identically regardless of what preceded it (cadence-agnostic guarantee).
  • Delta and dedup are one mechanism in the generic profile: an omitted row means "unchanged, keep it" (§10a.5).
  • Generic-profile header (§3.3) registers the delta fields: pack_root, key, delta, unchanged, base_root, new_root, count, savings.
  • New informative producer guidance (§10a.8): a producer MAY proactively emit a full payload on a schedule ("re-anchor") to bound accumulated-delta drift; cadence (default N=15 or an adaptive size-guard) is non-normative.

Grammar validation

The @id identity marker was selected empirically, not aesthetically. A targeted re-run of the 43-tokenizer barrier-merge method confirmed @id inside a field declaration {@id,...} merges 0.00% across 42 tokenizers (cleaner than bare @field at 4.4% and the general @ baseline at 1.09%), because the preceding { forces a token boundary. key= is required for explicit redundancy.

Comprehension validation

Generic-profile delta comprehension validated across a 50-turn session on ~10 models from 6+ vendors. Delta is safe at depth on 5 of 6 cleanly-measured models. The one mid-tier deep-drift edge case is closed by a producer-side periodic re-anchor (measured, reproduced across two runs; deep-turn accuracy 85% to 100%).

Reference implementations and conformance

  • All six SDKs implement §10a, each verified byte-for-byte against the shared fixtures: gcf-go, gcf-python, gcf-typescript, gcf-rust, gcf-swift, gcf-kotlin. Every implementation produces identical pack roots and delta wire (GenericPackRoot, DiffGenericSets, EncodeGenericFull / EncodeGenericDelta, DecodeGenericFull / DecodeGenericDelta, VerifyGenericDelta with atomic apply + root verification).
  • 12 shared conformance fixtures under tests/conformance/generic-pack-root/ (basic, nulls, string-keys, larger, number-formats) and tests/conformance/generic-delta/ (encode, verify-apply, root-mismatch, add-existing, decode-apply, empty-encode, verify-empty), driving four new runner operations: generic-pack-root, generic-delta, generic-delta-verify, generic-delta-decode.
  • Each SDK carries a self-proving suite (diff to encode to apply to recomputed root), full invariant/error coverage, malformed-wire-fails-closed, and the complete server to wire to consumer loop.
  • Tooling: tree-sitter-gcf v1.2.0 parses the @-prefixed identity field; the docs-site highlighter is updated to match.

Spec is now at v3.3.0. MIT, zero runtime dependencies.