feat: bootstrap accounts state #456
Conversation
Getting unregistered dreps in spdd warning
- Add BootstrapSnapshots type to pre-process raw snapshot data in publisher - Pass processed snapshots to accounts_state via AccountsBootstrapMessage - Simplify accounts_state bootstrap flow to match epochs_state pattern - Extract Arc ownership to avoid cloning large snapshot data - Add stake address stats logging for verification Note: utxo_value not populated from snapshot (requires UTxO aggregation)
lowhung
force-pushed
the
lowhung/388-bootstrap-accounts-module
branch
from
0fe69ed to
5a16940
Compare
There was a problem hiding this comment.
Pull request overview
This pull request implements bootstrap functionality for the accounts state module, enabling Acropolis to initialize from a Cardano ledger snapshot instead of syncing from genesis. The implementation follows the existing pattern established by epochs_state, with snapshot data being pre-processed by the snapshot publisher before being sent to accounts_state for initialization.
Key changes:
- Pre-processing of mark/set/go epoch snapshots in snapshot_bootstrapper before sending to accounts_state
- New
AccountsBootstrapMessagecontaining all data needed to initialize accounts state (stake addresses, pools, DReps, pots, and epoch snapshots) - Bootstrap flow in accounts_state that loads state from the message during startup when using snapshot method
- Migration of
Potstype to common for sharing between modules - Refactoring of snapshot parsing to use native types (
PoolRegistration,StakeAddress) instead of intermediate string representations - Cleanup of
DRepKeyHashnaming (previously inconsistently namedDrepKeyHash)
Reviewed changes
Copilot reviewed 18 out of 19 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
modules/accounts_state/src/accounts_state.rs |
Adds snapshot subscription and bootstrap waiting logic, handling AccountsBootstrapMessage |
modules/accounts_state/src/state.rs |
Implements bootstrap() method to load accounts, pools, DReps, pots, and epoch snapshots from message |
modules/accounts_state/src/snapshot.rs |
Adds from_bootstrap() method to convert pre-processed BootstrapSnapshot to internal Snapshot format |
modules/snapshot_bootstrapper/src/publisher.rs |
Implements AccountsCallback to publish processed bootstrap data; updates PoolCallback to send SPOState |
common/src/snapshot/mark_set_go.rs |
Adds BootstrapSnapshots types and from_raw() method to pre-process raw CBOR data into usable format |
common/src/snapshot/streaming_snapshot.rs |
Major refactoring: removes intermediate string types, adds wrapper types for clean CBOR decoding, implements AccountsCallback |
common/src/snapshot/pool_params.rs |
Deleted - replaced by using PoolRegistration directly with new snapshot decoders |
common/src/messages.rs |
Adds AccountsBootstrapMessage for passing pre-processed snapshot data |
common/src/types.rs |
Adds Pots struct (migrated from accounts_state) and fixes DRepKeyHash naming |
common/src/stake_addresses.rs |
Changes AccountState.stake_address from String to StakeAddress, adds Deserialize derives |
common/src/ledger_state.rs |
Adds SPOState::new() and extend() helper methods |
modules/spo_state/src/spo_state.rs |
Updates snapshot subscription to loop and handle multiple message types |
modules/governance_state/src/conway_voting_test.rs |
Fixes import order for DRepKeyHash (was DrepKeyHash) |
modules/accounts_state/Cargo.toml |
Removes unused serde dependency |
docs/streaming-snapshot-parser.md |
Updates documentation to reflect AccountsCallback rename |
common/examples/test_streaming_parser.rs |
Updates example to work with new callback signatures and types |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| match variant { | ||
| 0 => Ok(DRepCredential::AddrKeyhash(d.decode_with(ctx)?)), | ||
| 1 => Ok(DRepCredential::ScriptHash(d.decode_with(ctx)?)), | ||
| 0 => Ok(LocalDRepCredential(DRepCredential::ScriptHash( | ||
| d.decode_with(ctx)?, | ||
| ))), | ||
| 1 => Ok(LocalDRepCredential(DRepCredential::AddrKeyHash( | ||
| d.decode_with(ctx)?, | ||
| ))), |
There was a problem hiding this comment.
Critical bug: The variant order is reversed compared to the Credential enum definition. According to common/src/types.rs line 942-947, the Credential enum has ScriptHash as variant 0 and AddrKeyHash as variant 1. This decoder has them swapped, which will cause all DRep credentials to be decoded with the wrong type (scripts as keys and keys as scripts), corrupting the data.
The correct mapping should be:
- variant 0 →
DRepCredential::ScriptHash - variant 1 →
DRepCredential::AddrKeyHash
lowhung
force-pushed
the
lowhung/388-bootstrap-accounts-module
branch
from
72d28ef to
6fd31a4
Compare
This commit introduces several improvements to the snapshot parsing logic: - Simplify snapshot parsing by extracting common parsing logic - Add more descriptive logging for snapshot parsing - Create a new `ParsedSnapshotsContainer` to represent fully parsed snapshots - Reduce verbosity in logging and error handling
| ); | ||
|
|
||
| // Show snapshots info if available | ||
| if let Some(snapshots_info) = &metadata.snapshots { |
There was a problem hiding this comment.
I wasn't finding this information particularly useful, but happy to re-stage it if wanted.
| // Publish SPDD after bootstrap if bootstrap occurred | ||
| if let Some(block_info) = bootstrap_block_info { | ||
| let state = history.lock().await.get_current_state(); | ||
| let spdd = state.generate_spdd(); |
There was a problem hiding this comment.
This is an area of concern for me with this implementation. Would love some thoughts on whether we generate the spdd after the bootstrap here? As we discussed, utxos haven't been read and the address deltas have not been sent either.
We already have what is required to generate an spdd from the SnapshotSPO objects we send though. So maybe we don't even go through the stake_address map for this.
#456
| impl Snapshot { | ||
| /// Create a new snapshot from current stake address state (used at epoch boundary) | ||
| #[allow(clippy::too_many_arguments)] | ||
| pub fn new( |
There was a problem hiding this comment.
I lifted all of this from accounts_state/snapshot.rs. I feel bad adding even more into this type.rs file though.
Description
Implements bootstrap of accounts state module from snapshot data, enabling the accounts state to be initialized from a ledger snapshot rather than syncing from genesis.
Key changes:
Snapshottype moved toacropolis_common- eliminates duplicate type definitionsRegistrationChangeandRegistrationChangeKindtypes to common for tracking stake address registration changesRelated Issue(s)
Completes #388
How was this tested?
startup.method = "snapshot"(make run)➜ curl http://127.0.0.1:4340/accounts/stake1780j8ewwgup2vsx558zpn6jmryqt0pdevuqfh4dsfjmmgkg29pmrw { "utxo_value": 0, "rewards": 3516252, "delegated_spo": "pool1uj4u73qgtprqre78q75fq2vkcrpfrcdreqcqkvn6u0m2k6nk2yp", "delegated_drep": null }Checklist
Impact / Side effects
utxo_valueis not populated for stake addresses from snapshot (would require UTxO aggregation - known limitation)imbldependency toacropolis_commonReviewer notes / Areas to focus
Snapshottype incommon/src/types.rsnow includesnew(),from_raw(), and helper methodsSnapshotsContainerinmark_set_go.rs