Skip to content

spec: redesign on-chain layer to constant-per-batch footprint - #40

Merged
TaprootFreak merged 7 commits into
developfrom
spec/v2-constant-batch-footprint
Jun 6, 2026
Merged

spec: redesign on-chain layer to constant-per-batch footprint#40
TaprootFreak merged 7 commits into
developfrom
spec/v2-constant-batch-footprint

Conversation

@TaprootFreak

@TaprootFreak TaprootFreak commented Jun 6, 2026

Copy link
Copy Markdown
Contributor

Summary

Replaces the per-spender Variant-1 on-chain design with the constant-per-batch design from the Shielded CSV paper:

  • On-chain (new): one 231-byte fixed-size BatchInscription per publisher batch (~58 vBytes amortised), regardless of how many records it covers. Per-record marginal on-chain cost is zero.
  • Off-chain (new): the BatchBundle (k=3 replicated by the same DA discipline as CoinProof bundles, content-addressed by Hc("BatchBundle", serialize)) and the AggregateBatchProof (recursive PCD attesting per-member validity, nullifier-set integrity, and new_root = SMT.insert_many(prev_root, nfs)).
  • SpendRecord stays as the per-spender object but moves off-chain; a spender hands it to a publisher, which aggregates many into a BatchBundle.
  • Mints are off-chain unless the issuer chooses to anchor them.

Why

The Variant-1 design (PR #21) traded constant-per-batch blockspace for blind-node rebuildability — at ~40 vBytes per record on Bitcoin. Discussion revealed the rebuildability property is preserved at lower cost by extending the existing k=3 replication discipline to BatchBundles: a verifier still verifies trustlessly (recursive proof check, not a trust assumption), the on-chain root commitment plus DA-replicated bundles deliver the same property as raw on-chain nullifiers. At 10 sat/vB and $100k BTC: ~$0.58 per batch (constant); per-record amortised well under $0.01 for typical batch sizes.

What changed

End-to-end rewrites: §3.1 on-chain object, §3.2 publisher signing, §3.3 off-chain signature handling, §3.4 publisher role, §3.5 inscription format, §3.6 chain scanning, §3.7 nullifier accumulator, §3.8 fees, §3.10 transaction states.

Cross-section updates: §1.4 SpendRecord off-chain + new BatchInscription/BatchBundle entries; §1.6 trees framing; §2.1 clause 4; §2.2 adds AggregateBatchProof; §2.3.1 mint off-chain; §2.3.2 send hands record to publisher; §2.3.3 receive non-membership via Path A or Path B; §4.5 recovery step 2; §4.6 k=3 extended to BatchBundles; §5.6 public-mode explorer; §5.7 balance attestation re-anchored via batch; §6.1 node responsibilities; §6.3 portability argument; §6.4 interfaces; glossary; test vectors V.5/V.6.

Diff: +315 / -241 in a single file. Build is clean (no broken anchors).

Test plan

  • 3-subagent review loop (Quality + Logic) — see if either reviewer finds inconsistencies between the rewritten §3 and the cross-references in §1, §2, §4, §5, §6.
  • Verify the Mint case is internally consistent (mints are off-chain unless optionally batched).
  • Verify the BalanceAttestation rewrite (§5.7) correctly anchors via BatchBundle position + BatchInscription completion.
  • Verify the on-chain byte layout (V.6) matches §3.5 word for word.
  • Verify the AggregateBatchProof public inputs in §2.2 match the chain-scanning verification step in §3.6.

Replace the per-spender Variant-1 design (each SpendRecord inscribed on
Bitcoin with raw nullifiers in the clear, ~40 vBytes per record marginal)
with the constant-per-batch design from the Shielded CSV paper.

What is on-chain (new):
  - A 231-byte fixed-size BatchInscription per publisher batch, regardless
    of how many records it covers: { publisher_pubkey, prev_root, new_root,
    bundle_locator, block_anchor, signature }. ~58 vBytes amortised per
    batch. Per-record marginal on-chain cost is zero.

What is off-chain (new):
  - The BatchBundle, k=3 replicated by the same DA discipline as CoinProof
    bundles: { prev_root, new_root, nullifiers, spend_records,
    aggregate_proof }. Content-addressed by Hc("BatchBundle", serialize).
  - The AggregateBatchProof, a recursive PCD proof from the publisher's
    batch-aggregation circuit C_batch attesting per-member validity,
    nullifier-set integrity, and new_root = SMT.insert_many(prev_root, nfs).

SpendRecord stays as the per-spender object but is now off-chain: a spender
hands it to a publisher, the publisher aggregates many into a BatchBundle.
Mints are off-chain unless an issuer chooses to anchor them.

Verification paths (both trustless):
  - Path A: maintain the accumulator by following BatchInscriptions and
    verifying each AggregateBatchProof.
  - Path B (light client): follow only inscribed roots, ask any Path-A
    node for a self-verifying SMT path against the on-chain new_root.

Cost impact:
  - Per-batch on-chain: ~$0.58 at 10 sat/vB + $100k BTC (was variable,
    grew linearly).
  - Per-record amortised: well below $0.01 for typical batch sizes.

Sections rewritten end-to-end: §3.1 on-chain object, §3.2 publisher
signing, §3.3 off-chain signature handling, §3.4 publisher role,
§3.5 inscription format, §3.6 chain scanning, §3.7 nullifier accumulator,
§3.8 fees, §3.10 transaction states.

Cross-section updates: §1.4 SpendRecord moved off-chain + new BatchInscription
and BatchBundle entries; §1.6 trees framing; §2.1 clause 4; §2.2 adds
AggregateBatchProof type; §2.3.1 mint off-chain; §2.3.2 send hands record
to publisher; §2.3.3 receive non-membership Path A/B; §4.5 step 2 rebuild;
§4.6 k extended to BatchBundles; §5.6 public-mode explorer; §5.7 balance
attestation re-anchored via batch; §6.1 node responsibilities; §6.3 portability
argument; §6.4 interfaces; glossary updates; test vectors V.5/V.6.
Quality (blockers):
- Replace residual 'on-chain SpendRecord' phrases with 'BatchInscription'
  in §5 (lines 1039, 1057, 1211, 1411, 1474). SpendRecord is off-chain.
- §5.6 step 3 / §5.7 verifier flow / §5.8 history flow: re-route the
  receiver's anchor lookup through the containing BatchBundle, not
  directly to a "SpendRecord state".
- Glossary: add Pkₚ, C_batch, Path A, Path B entries.

Quality (important):
- Glossary alphabetical sort fix (BatchBundle/BatchInscription moved
  after balances; asset_id repositioned).
- Six §3.7/§3.10 references switched from #3--on-chain-layer (section
  anchor) to deep anchors (#37-the-nullifier-accumulator,
  #310-transaction-states) for consistency.
- §1.4 SpendRecord adds explicit u8 k length-prefix field; size
  estimate updated to 161 + 32·|nf| (matches V.5 test vector).

Logic (important):
- B1: AggregateBatchProof now binds bundle_locator as a public input
  (§2.2 clauses 5–6). C_batch attests
  Hc("BatchBundle", serialize(BatchBundle)) == bundle_locator over a
  fixed canonical preimage (prev_root ‖ new_root ‖ u32-be(m) ‖
  member SpendRecords). §3.6 step 7 updated to reflect new public-input
  triple (prev_root, new_root, bundle_locator).
- B2: §3.7 Path B trustlessness explicitly covers both inclusion and
  exclusion paths against the same on-chain new_root.
- B3: §3.7 reorg paragraph spells out the cascade: revert of batch N
  invalidates every later batch whose prev_root was N.new_root.
- B8: §4.6 normative long-term retention rule: every node that admits
  a BatchInscription MUST retain its BatchBundle indefinitely
  (or verify k=3 peer copies before pruning). Practical replica count
  grows monotonically with the scanning network.

Logic (nits):
- Light-client pending sub-state distinction: §3.7 Path B explains how
  a light client tells confirmation-pending from DA-pending via the
  serving node.
- Unbatched mints: §3.10 introduces an explicit `mint-verified` status
  for issuer-chose-not-to-anchor mints, distinct from
  completed/pending/failed.
- §3.4 publisher: explicit stale-bundle handling (re-batch within
  6 blocks or release records back; spender MAY re-submit; stale
  bundles not subject to k=3 DA).
- §2.2 clause 5: network/chain separation embedded in C_batch verifier
  data (mainnet vs testnet tags).
- V.6 test vector: block_anchor.height marked as <REGEN — illustrative>
  to avoid hard-coded value.
L1 (important) — Bundle-locator preimage clarity:
  §1.4 BatchBundle entry now explicitly declares the top-level
  `nullifiers` field as a *derived view* (multi-set concatenation of
  member SpendRecord nullifiers, bound by §2.2 clauses 3-4) and pins
  the canonical `serialize(BatchBundle)` preimage as
  `prev_root ‖ new_root ‖ u32-be(m) ‖ SpendRecord₁ ‖ … ‖ SpendRecord_m`
  with both the derived `nullifiers` view and the `aggregate_proof`
  excluded. §3.6 Step 6 now references this canonical preimage and
  notes the separate S2C binding of `aggregate_proof`.

L2 (important) — `mint-verified` propagated across all receiver paths:
  §3.10 "Receivers SHALL act only on `completed`" now adds the
  `mint-verified` exception with the rule that explorers MUST render
  it distinct from `completed`. §5.6 step 3 adds the non-batched-mint
  branch (InitialProof re-verification → `mint-verified`). §5.8 history
  flow adds the same branch.

L3 (nit) — §3.6 Step 6 now points at §2.2 clause 6 for the
  `aggregate_proof`/`bundle_locator` separation, removing the
  apparent double-binding confusion.

Quality round 2: PASS_CLEAN, no changes needed.
N1 — Glossary BatchBundle entry now lists `nullifiers` as `derived view`
  and pins the canonical preimage explicitly, matching §1.4 and §2.2 Cl. 6.

N2 — New glossary entry `mint-verified (transaction state)` added between
  `Mint` and `MMR`; `Transaction state` cross-ref updated to include it.

N3 (optional) — §3.2 line 741: replaced "over the whole bundle" with
  "over the canonical serialisation of the bundle per §1.4, which excludes
  the `aggregate_proof` field by construction" — makes the dual-binding
  argument explicit rather than relying on the reader's inference.

Quality round 2: PASS_CLEAN, no changes needed.
Logic round 3: 0 blockers, 2 nits + 1 optional nit, all addressed here.
PRs #34 (DetectKey rename), #36 (ACK nonce-bind), #38 (Invoice addr_sig)
auto-merge into V2 sections they don't touch (§1.1/§1.3/§4.2/§4.3/glossary).

PRs #35 (rollback semantics for duplicate-nf in §3.6) and #37 (precise
anchors throughout) conflict with the V2 redesign — V2 wins by -X ours,
because:
- #35 fixes V1's chain-rebuild rollback path; V2 has no on-chain
  nullifier list to rebuild — accumulator advances via inscribed
  prev_root -> new_root attested by AggregateBatchProof, so the
  duplicate-nf semantics are caught in-circuit by §2.2 clause 3.
- #37's anchor polish targeted V1 section structure (§3.2
  SpendRecord-signing etc.); V2 renamed those sections so the polished
  anchors point at obsolete IDs.

PR #39 (latest-state selection rule for multi-node, §6.3) is re-added
below in V2-adapted form (BatchInscription instead of SpendRecord).
The merge of develop brought in two paragraphs that were written against
the V1 on-chain-SpendRecord model:

- §4.3 (#38 addr_sig): "observes Pk₀ from an on-chain SpendRecord" — in
  V2 the SpendRecord is off-chain inside the publisher's BatchBundle,
  which is content-addressed and k=3-replicated, so Pk₀ is still publicly
  observable but via the bundle, not the chain. Same threat surface, same
  fix (addr_sig closes it), updated wording.

- §6.3 (#39 latest-state selection): "anchoring SpendRecord is in state
  completed" — V2's anchor of state is the BatchInscription, not the
  SpendRecord (which inherits the state of the BatchInscription it was
  batched into per §3.10).

No semantic change to either rule; just terminology brought in line with
the V2 on-chain object naming.
@TaprootFreak

TaprootFreak commented Jun 6, 2026

Copy link
Copy Markdown
Contributor Author

Re-review (doc-consistency pass)

The PR is now MERGEABLE/CLEAN with all CI green, and the body correctly V2-adapts the inherited #38/#39 changes (the F9 three-check at §4.3 now reads "Pk₀ from the k = 3-replicated BatchBundle" rather than on-chain, and the #37 profile-preimage pin survived). Two doc-level issues remain that CI cannot catch:

1. Blocking — glossary Invoice entry contradicts the body (F9 regression from the develop-merge)

The develop-merge reverted the glossary Invoice entry to its pre-#38 shape, dropping pk0 and addr_sig:

this PR (glossary): {amount, recipient, asset_id, memo?, ivpk, op_pubkey, relays, sig}
develop (glossary): {amount, recipient, asset_id, memo?, pk0, ivpk, op_pubkey, relays, addr_sig, sig}   + "Both are required."

The body (§1.5 schema + §4.3 three-check) declares pk0 and addr_sig mandatory and describes the forgery protection they provide, so the glossary now lists an Invoice tuple that is missing two fields the spec requires. onBrokenAnchors/CodeQL do not detect a glossary↔body contradiction. Please re-sync the glossary Invoice entry to the develop (post-#38) form.

2. Minor — anchor-tightening drift vs the #37 convention

27 links whose label cites a specific §X.Y now resolve to a section-top anchor (#1--foundations-normative, #3--on-chain-layer, #5--access--explorer, …), undoing the #37 tightening convention that is now on develop. The anchors are valid (build passes); they are just less precise than the labels claim. A pass re-pointing these 27 to their §X.Y subsection anchors restores the convention.

Two doc-level findings from the post-merge re-review on PR #40:

1. Blocker — glossary Invoice entry was reverted to its pre-#38 form by
   the develop-merge (dropped pk0 and addr_sig fields), contradicting
   §1.5 and §4.3 which still mandate them. Re-synced the glossary entry
   to develop's post-#38 form (full tuple {amount, recipient, asset_id,
   memo?, pk0, ivpk, op_pubkey, relays, addr_sig, sig} + Both-required
   sentence).

2. Anchor drift — about 30 cross-references with §X.Y labels resolved
   to section-top anchors (#1--foundations-normative, #3--on-chain-layer,
   etc.), undoing the #37 tightening convention now on develop. Tightened
   all to their subsection anchors:
   - [Foundations §1.X] -> #11-cryptographic-primitives / #12-key-hierarchy
     / #13-per-coin-keys-... / #14-identifiers-and-hashes /
     #15-core-data-structures / #16-trees-... / #17-encoding-...
   - [Proofs §2.X] -> #22-proof-types / #233-receive
   - [On-chain §3.X] -> #31-the-on-chain-object
   - [Access & Explorer §5.X] -> #51-capability-gated-pull /
     #56-shareable-confirmation-links
   - Same for the "[Foundations](#1--) §1.X" plain-text-after-link pattern.

Build passes (no broken anchors).
@TaprootFreak

Copy link
Copy Markdown
Contributor Author

Both findings addressed in a0dd748:

  1. Glossary Invoice entry — re-synced to develop's post-spec: add addr_sig (sk₀) + pk0 binding to Invoice (F9) #38 form (full tuple with pk0/addr_sig + "Both are required." sentence).
  2. Anchor drift — 33 cross-references retightened. The Foundations §1.X family now resolves to docs: replace stale SP1/zkVM references with Plonky2; fix mock-mode and polling claims #11/docs: replace stale SP1/zkVM references with Plonky2; fix mock-mode and polling claims #12/docs: add Trust Model page + reframe trust-model section as run-your-own-node #13/Release: develop -> main #14/docs: point footer source link to the Plonky2 circuit (drop stale SP1 link) #15/Release: develop -> main #16/Discoverability: unblock AI crawlers, add llms.txt, host protocol docs on docs.zkcoins.com #17 subsections, Proofs §2.3.3 to #233-receive, Access & Explorer §5.6/§5.7 to Spec: global serial nullifier accumulator forces publisher contention — throughput / centralization economics unanalyzed (§3.4/§3.6) #56-shareable-confirmation-links, etc. Also caught the [Foundations](#1--) §1.X plain-text-after-link variant.

Build passes locally with no broken anchors. Re-review pass welcome.

The architectural review on the constant-per-batch model + AggregateBatchProof + off-chain accumulator-DA trade-off — separate from these doc-consistency fixes — remains for a human cryptographer / paper-fidelity pass before merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant