spec: redesign on-chain layer to constant-per-batch footprint - #40
Conversation
Replace the per-spender Variant-1 design (each SpendRecord inscribed on
Bitcoin with raw nullifiers in the clear, ~40 vBytes per record marginal)
with the constant-per-batch design from the Shielded CSV paper.
What is on-chain (new):
- A 231-byte fixed-size BatchInscription per publisher batch, regardless
of how many records it covers: { publisher_pubkey, prev_root, new_root,
bundle_locator, block_anchor, signature }. ~58 vBytes amortised per
batch. Per-record marginal on-chain cost is zero.
What is off-chain (new):
- The BatchBundle, k=3 replicated by the same DA discipline as CoinProof
bundles: { prev_root, new_root, nullifiers, spend_records,
aggregate_proof }. Content-addressed by Hc("BatchBundle", serialize).
- The AggregateBatchProof, a recursive PCD proof from the publisher's
batch-aggregation circuit C_batch attesting per-member validity,
nullifier-set integrity, and new_root = SMT.insert_many(prev_root, nfs).
SpendRecord stays as the per-spender object but is now off-chain: a spender
hands it to a publisher, the publisher aggregates many into a BatchBundle.
Mints are off-chain unless an issuer chooses to anchor them.
Verification paths (both trustless):
- Path A: maintain the accumulator by following BatchInscriptions and
verifying each AggregateBatchProof.
- Path B (light client): follow only inscribed roots, ask any Path-A
node for a self-verifying SMT path against the on-chain new_root.
Cost impact:
- Per-batch on-chain: ~$0.58 at 10 sat/vB + $100k BTC (was variable,
grew linearly).
- Per-record amortised: well below $0.01 for typical batch sizes.
Sections rewritten end-to-end: §3.1 on-chain object, §3.2 publisher
signing, §3.3 off-chain signature handling, §3.4 publisher role,
§3.5 inscription format, §3.6 chain scanning, §3.7 nullifier accumulator,
§3.8 fees, §3.10 transaction states.
Cross-section updates: §1.4 SpendRecord moved off-chain + new BatchInscription
and BatchBundle entries; §1.6 trees framing; §2.1 clause 4; §2.2 adds
AggregateBatchProof type; §2.3.1 mint off-chain; §2.3.2 send hands record
to publisher; §2.3.3 receive non-membership Path A/B; §4.5 step 2 rebuild;
§4.6 k extended to BatchBundles; §5.6 public-mode explorer; §5.7 balance
attestation re-anchored via batch; §6.1 node responsibilities; §6.3 portability
argument; §6.4 interfaces; glossary updates; test vectors V.5/V.6.
Quality (blockers): - Replace residual 'on-chain SpendRecord' phrases with 'BatchInscription' in §5 (lines 1039, 1057, 1211, 1411, 1474). SpendRecord is off-chain. - §5.6 step 3 / §5.7 verifier flow / §5.8 history flow: re-route the receiver's anchor lookup through the containing BatchBundle, not directly to a "SpendRecord state". - Glossary: add Pkₚ, C_batch, Path A, Path B entries. Quality (important): - Glossary alphabetical sort fix (BatchBundle/BatchInscription moved after balances; asset_id repositioned). - Six §3.7/§3.10 references switched from #3--on-chain-layer (section anchor) to deep anchors (#37-the-nullifier-accumulator, #310-transaction-states) for consistency. - §1.4 SpendRecord adds explicit u8 k length-prefix field; size estimate updated to 161 + 32·|nf| (matches V.5 test vector). Logic (important): - B1: AggregateBatchProof now binds bundle_locator as a public input (§2.2 clauses 5–6). C_batch attests Hc("BatchBundle", serialize(BatchBundle)) == bundle_locator over a fixed canonical preimage (prev_root ‖ new_root ‖ u32-be(m) ‖ member SpendRecords). §3.6 step 7 updated to reflect new public-input triple (prev_root, new_root, bundle_locator). - B2: §3.7 Path B trustlessness explicitly covers both inclusion and exclusion paths against the same on-chain new_root. - B3: §3.7 reorg paragraph spells out the cascade: revert of batch N invalidates every later batch whose prev_root was N.new_root. - B8: §4.6 normative long-term retention rule: every node that admits a BatchInscription MUST retain its BatchBundle indefinitely (or verify k=3 peer copies before pruning). Practical replica count grows monotonically with the scanning network. Logic (nits): - Light-client pending sub-state distinction: §3.7 Path B explains how a light client tells confirmation-pending from DA-pending via the serving node. - Unbatched mints: §3.10 introduces an explicit `mint-verified` status for issuer-chose-not-to-anchor mints, distinct from completed/pending/failed. - §3.4 publisher: explicit stale-bundle handling (re-batch within 6 blocks or release records back; spender MAY re-submit; stale bundles not subject to k=3 DA). - §2.2 clause 5: network/chain separation embedded in C_batch verifier data (mainnet vs testnet tags). - V.6 test vector: block_anchor.height marked as <REGEN — illustrative> to avoid hard-coded value.
L1 (important) — Bundle-locator preimage clarity: §1.4 BatchBundle entry now explicitly declares the top-level `nullifiers` field as a *derived view* (multi-set concatenation of member SpendRecord nullifiers, bound by §2.2 clauses 3-4) and pins the canonical `serialize(BatchBundle)` preimage as `prev_root ‖ new_root ‖ u32-be(m) ‖ SpendRecord₁ ‖ … ‖ SpendRecord_m` with both the derived `nullifiers` view and the `aggregate_proof` excluded. §3.6 Step 6 now references this canonical preimage and notes the separate S2C binding of `aggregate_proof`. L2 (important) — `mint-verified` propagated across all receiver paths: §3.10 "Receivers SHALL act only on `completed`" now adds the `mint-verified` exception with the rule that explorers MUST render it distinct from `completed`. §5.6 step 3 adds the non-batched-mint branch (InitialProof re-verification → `mint-verified`). §5.8 history flow adds the same branch. L3 (nit) — §3.6 Step 6 now points at §2.2 clause 6 for the `aggregate_proof`/`bundle_locator` separation, removing the apparent double-binding confusion. Quality round 2: PASS_CLEAN, no changes needed.
N1 — Glossary BatchBundle entry now lists `nullifiers` as `derived view` and pins the canonical preimage explicitly, matching §1.4 and §2.2 Cl. 6. N2 — New glossary entry `mint-verified (transaction state)` added between `Mint` and `MMR`; `Transaction state` cross-ref updated to include it. N3 (optional) — §3.2 line 741: replaced "over the whole bundle" with "over the canonical serialisation of the bundle per §1.4, which excludes the `aggregate_proof` field by construction" — makes the dual-binding argument explicit rather than relying on the reader's inference. Quality round 2: PASS_CLEAN, no changes needed. Logic round 3: 0 blockers, 2 nits + 1 optional nit, all addressed here.
PRs #34 (DetectKey rename), #36 (ACK nonce-bind), #38 (Invoice addr_sig) auto-merge into V2 sections they don't touch (§1.1/§1.3/§4.2/§4.3/glossary). PRs #35 (rollback semantics for duplicate-nf in §3.6) and #37 (precise anchors throughout) conflict with the V2 redesign — V2 wins by -X ours, because: - #35 fixes V1's chain-rebuild rollback path; V2 has no on-chain nullifier list to rebuild — accumulator advances via inscribed prev_root -> new_root attested by AggregateBatchProof, so the duplicate-nf semantics are caught in-circuit by §2.2 clause 3. - #37's anchor polish targeted V1 section structure (§3.2 SpendRecord-signing etc.); V2 renamed those sections so the polished anchors point at obsolete IDs. PR #39 (latest-state selection rule for multi-node, §6.3) is re-added below in V2-adapted form (BatchInscription instead of SpendRecord).
The merge of develop brought in two paragraphs that were written against the V1 on-chain-SpendRecord model: - §4.3 (#38 addr_sig): "observes Pk₀ from an on-chain SpendRecord" — in V2 the SpendRecord is off-chain inside the publisher's BatchBundle, which is content-addressed and k=3-replicated, so Pk₀ is still publicly observable but via the bundle, not the chain. Same threat surface, same fix (addr_sig closes it), updated wording. - §6.3 (#39 latest-state selection): "anchoring SpendRecord is in state completed" — V2's anchor of state is the BatchInscription, not the SpendRecord (which inherits the state of the BatchInscription it was batched into per §3.10). No semantic change to either rule; just terminology brought in line with the V2 on-chain object naming.
Re-review (doc-consistency pass)The PR is now 1. Blocking — glossary
|
Two doc-level findings from the post-merge re-review on PR #40: 1. Blocker — glossary Invoice entry was reverted to its pre-#38 form by the develop-merge (dropped pk0 and addr_sig fields), contradicting §1.5 and §4.3 which still mandate them. Re-synced the glossary entry to develop's post-#38 form (full tuple {amount, recipient, asset_id, memo?, pk0, ivpk, op_pubkey, relays, addr_sig, sig} + Both-required sentence). 2. Anchor drift — about 30 cross-references with §X.Y labels resolved to section-top anchors (#1--foundations-normative, #3--on-chain-layer, etc.), undoing the #37 tightening convention now on develop. Tightened all to their subsection anchors: - [Foundations §1.X] -> #11-cryptographic-primitives / #12-key-hierarchy / #13-per-coin-keys-... / #14-identifiers-and-hashes / #15-core-data-structures / #16-trees-... / #17-encoding-... - [Proofs §2.X] -> #22-proof-types / #233-receive - [On-chain §3.X] -> #31-the-on-chain-object - [Access & Explorer §5.X] -> #51-capability-gated-pull / #56-shareable-confirmation-links - Same for the "[Foundations](#1--) §1.X" plain-text-after-link pattern. Build passes (no broken anchors).
|
Both findings addressed in a0dd748:
Build passes locally with no broken anchors. Re-review pass welcome. The architectural review on the constant-per-batch model + AggregateBatchProof + off-chain accumulator-DA trade-off — separate from these doc-consistency fixes — remains for a human cryptographer / paper-fidelity pass before merge. |
Summary
Replaces the per-spender Variant-1 on-chain design with the constant-per-batch design from the Shielded CSV paper:
BatchInscriptionper publisher batch (~58 vBytes amortised), regardless of how many records it covers. Per-record marginal on-chain cost is zero.BatchBundle(k=3 replicated by the same DA discipline asCoinProofbundles, content-addressed byHc("BatchBundle", serialize)) and theAggregateBatchProof(recursive PCD attesting per-member validity, nullifier-set integrity, andnew_root = SMT.insert_many(prev_root, nfs)).BatchBundle.Why
The Variant-1 design (PR #21) traded constant-per-batch blockspace for blind-node rebuildability — at ~40 vBytes per record on Bitcoin. Discussion revealed the rebuildability property is preserved at lower cost by extending the existing
k=3replication discipline toBatchBundles: a verifier still verifies trustlessly (recursive proof check, not a trust assumption), the on-chain root commitment plus DA-replicated bundles deliver the same property as raw on-chain nullifiers. At 10 sat/vB and $100k BTC: ~$0.58 per batch (constant); per-record amortised well under $0.01 for typical batch sizes.What changed
End-to-end rewrites: §3.1 on-chain object, §3.2 publisher signing, §3.3 off-chain signature handling, §3.4 publisher role, §3.5 inscription format, §3.6 chain scanning, §3.7 nullifier accumulator, §3.8 fees, §3.10 transaction states.
Cross-section updates: §1.4 SpendRecord off-chain + new BatchInscription/BatchBundle entries; §1.6 trees framing; §2.1 clause 4; §2.2 adds AggregateBatchProof; §2.3.1 mint off-chain; §2.3.2 send hands record to publisher; §2.3.3 receive non-membership via Path A or Path B; §4.5 recovery step 2; §4.6 k=3 extended to BatchBundles; §5.6 public-mode explorer; §5.7 balance attestation re-anchored via batch; §6.1 node responsibilities; §6.3 portability argument; §6.4 interfaces; glossary; test vectors V.5/V.6.
Diff: +315 / -241 in a single file. Build is clean (no broken anchors).
Test plan