Pre-populate the shielded pool with large N (e.g. 1M notes) at devnet genesis for sync/scale testing

[QuantumExplorer]

Expected Behavior

We want to be able to boot a devnet whose Orchard shielded pool already contains a large, configurable number of notes (target: 1,000,000), with a known owned subset belonging to a test wallet, so we can realistically benchmark and exercise:

• Client wallet sync at scale — chunked parallel fetch + proof verification + trial-decryption + local commitment-tree (shardtree) build over ~1M notes (~489 chunks of 2048).
• Platform-side query / proof scaling — ShieldedEncryptedNotes range queries, BulkAppendTree chunk proofs, anchor lookups against a deep tree.
• Spend correctness against a deep tree — a spend from the owned subset must witness and validate against a recorded anchor.

Bringing the devnet up with this state should take minutes, be deterministic/repeatable (fixed RNG seed → identical GroveDB root hash), and require zero runtime proving.

Current Behavior

The only way to add a note to the pool today is by submitting a shielded state transition (Shield / ShieldFromAssetLock / Transfer / Unshield / Withdraw). Each one:

• requires a Halo 2 proof (~seconds of CPU to produce, plus verification on every validator), and
• only adds 1–2 notes (the action outputs).

So reaching 1M notes through the normal path is effectively infeasible — on the order of days-to-weeks of proving plus the orchestration of hundreds of thousands of transitions. There is no seeding/bootstrap path for the shielded pool today, so large-N performance work is currently blocked.

Key insight that unblocks this: a Halo 2 proof is only needed to pass consensus validation. The thing that actually mutates the tree — Drive::insert_note_op → commitment_tree_insert_op / BulkAppend — is proof-free. There is already precedent for appending notes directly at the Drive layer in tests (packages/rs-drive/src/drive/shielded/notes_count/v0/mod.rs loops insert_note_op). On a devnet, where we own the nodes and genesis, we can populate the tree directly and skip proving entirely.

Possible Solution

Bake the shielded-pool GroveDB state into the devnet genesis / initial app state (option "B" from the design discussion).

Background — what the pool looks like on disk

• Pool root: [ShieldedBalances, "M"] with subtrees (packages/rs-drive/src/drive/shielded/paths.rs):
  • [128] notes — a composite CommitmentTree = a BulkAppendTree (two-level: a DenseFixedSizedMerkleTree buffer + a chunk MMR of immutable chunk blobs; chunk_power = 11 → 2048 notes/chunk) plus a Sinsemilla frontier (~1 KB, under COMMITMENT_TREE_DATA_KEY) that yields the Orchard anchor.
  • [192] anchors-in-pool and [96] anchors-by-height — the provable anchor history used by validate_anchor_exists at spend time (recorded per block-when-changed by record_shielded_pool_anchor_if_changed).
  • [32] total-balance sum item.
• A note entry in the BulkAppendTree is cmx(32) || rho(32) || encrypted_note.
• Element::BulkAppendTree(total_count, chunk_power, flags) holds total_count in the parent Merk; mmr_size lives in the BulkAppendTree metadata key.

What a seeded note needs to be valid

• cmx must be a valid Pallas field element (merkle_hash_from_bytes rejects non-elements).
• rho/nullifier — any 32 bytes for filler.
• encrypted_note — any blob, but use a realistic Orchard length (~600 B) so the sync wire/memory profile is representative.

Two tiers of seeded notes

1. Filler (the bulk of the 1M) — random valid cmx + random realistic-length ciphertext. The wallet trial-decrypts and fails fast on each (exercises the full fetch → verify → decrypt-miss → tree-append loop). Not owned, not spendable.
2. Owned subset (a known few hundred/thousand, at known positions) — real Orchard notes generated to the test wallet's address: true cmx, encrypted to its IVK, matching nullifier. Note encryption is cheap — only the spend/output proof is expensive, and that's only paid for the handful actually spent later. These make the test wallet show a real balance and be spendable.

Implementation sketch

1. Offline generator (deterministic via a fixed seed) that produces N note entries: tier-1 filler interleaved with tier-2 owned notes at recorded positions. Inputs: total N, owned-subset spec (count + target wallet IVK/address + values), ciphertext size, chunk_power, RNG seed.
2. Compute the full BulkAppendTree state from those entries: completed chunk blobs (e{u64} keys), tail buffer entries (b{u32}), chunk-MMR nodes (m{u64}), metadata (M, mmr_size), and the state_root (blake3("bulk_state" || mmr_root || dense_tree_root)).
3. Compute the commitment-tree Sinsemilla frontier over all N cmx (the COMMITMENT_TREE_DATA_KEY blob) → the resulting anchor.
4. Write the Element::CommitmentTree(total_count = N, chunk_power = 11, flags) at [128] with the computed child hash, plus the BulkAppendTree data and frontier blob.
5. Record the anchor into [192]/[96] so spends from the owned subset validate; optionally set [32] total-balance sum item consistent with the owned value.
6. Inject this into the devnet genesis GroveDB / platform init (dashmate devnet config) so the node boots with a consistent root hash.

Config knobs

total_notes (incl. 1,000,000), owned_count + target wallet, ciphertext_size, chunk_power (match the active platform version), and an rng_seed for reproducibility.

Acceptance criteria

• A devnet can be brought up with a configurable N (including 1M) notes in the shielded pool, with a consistent, reproducible GroveDB root hash.
• ShieldedEncryptedNotes range queries return proven chunks against the seeded tree.
• A test wallet with the owned IVK syncs and shows the expected balance from the owned subset.
• An owned note is spendable (anchor recorded; a real proof produced only at spend time).
• Generation + boot completes in minutes, not days.

The one real cost on this path

Computing the commitment-tree Sinsemilla frontier over N cmx (~N Pallas hashes → minutes for 1M). The BulkAppendTree side (chunk blobs + MMR) is blake3 and fast. This is dramatically cheaper than ~1M Halo 2 proofs.

Alternatives Considered

• A — drive-abci batch seeder at runtime. Extend the strategy_tests harness (it already drives Drive directly) to push notes in large GroveDB batches via the BulkAppend op (which already groups appends by path/key in preprocessing), 2048-aligned so chunks finalize. Simpler to write and reuses existing infra, but it's a runtime populate (slower per devnet bring-up) and not a reusable genesis image. Good as a stepping stone toward B.
• C — one-shot devnet seed binary/RPC. A tool that opens GroveDB and bulk-appends once per bring-up. Similar tradeoffs to A; not reproducible-by-image.
• Naive real-transition path (rejected). Submitting ~1M real shielded transitions with Halo 2 proofs — days/weeks, infeasible.

B is preferred for repeatable benchmarking: the seeded state becomes a reusable genesis image with a fixed root hash, zero per-run cost, and no runtime proving.

Additional Context

Relevant code:

• Proof-free note insert: packages/rs-drive/src/drive/shielded/insert_note/ (insert_note_op / insert_note_op_v0 → commitment_tree_insert_op).
• Seeding precedent: packages/rs-drive/src/drive/shielded/notes_count/v0/mod.rs (loops insert_note_op).
• Pool layout: packages/rs-drive/src/drive/shielded/paths.rs.
• Anchor recording / validation: packages/rs-drive/src/drive/shielded/record_anchor_if_changed/, validate_anchor_exists.
• BulkAppendTree design: grovedb book chapter bulk-append-tree.md + grovedb-bulk-append-tree/.
• Client sync (the consumer being benchmarked): packages/rs-sdk/src/platform/shielded/notes_sync/sync_shielded_notes.rs (chunked, 2048-aligned, proof per chunk; max_encrypted_notes_per_query = 2048) and packages/rs-platform-wallet/src/wallet/shielded/sync.rs (sync_notes_across).

Scope: devnet only — no mainnet/testnet seeding, no consensus-rule changes.