ERA file consumer and producer via LCLI#9273
Open
dapplion wants to merge 15 commits into
Open
Conversation
Test vectors are now hosted at dapplion/era-test-vectors and downloaded via Makefile (same pattern as slashing_protection interchange tests).
- Add docs to EraFileDir, import_all, and module-level usage example - Rename let _span to let _ for debug spans - Remove unused _start_slot variable - Extract parse_era_filename with unit tests - Add rejects_wrong_trusted_slot test
EraFileDir::new now takes genesis_validators_root and EraImportTrust: - TrustedStateRoot(era_number, root): uses that ERA as reference, verifies its state root, imports only ERAs 0..=era_number - Untrusted: uses highest ERA in directory as reference Trust checks (genesis_validators_root, state root) moved from import_all/import_era_file into EraFileDir::new. Removed all expects/unwraps from production code.
Add `init_genesis_store` + `advance_store_to_era` so that after ERA import the store metadata (split, anchor, fork choice) is fully set up for the regular `resume_from_db` → `build()` startup path. Key changes in consumer.rs: - `init_genesis_store`: standalone genesis init (block, state, anchor, fork choice) - `advance_store_to_era`: advances split/anchor/fork choice to ERA boundary - `write_state_root_index_for_era`: writes both BeaconStateRoots (slot→root) and BeaconColdStateSummary (root→slot) for every slot - Uses `from_persisted` instead of `get_forkchoice_store` to avoid deriving a wrong anchor block root from the ERA boundary state's latest_block_header Test `chain_boots_from_imported_db` verifies: - canonical_head matches expected head root - Every slot's state is accessible via state_root_at_slot → get_state - Every slot's block is accessible via block_root_at_slot → get_blinded_block - Blocks form a valid parent chain (parent_root linkage) Also fixes producer to use get_blinded_block + make_full_block for cold blocks where get_full_block fails when prune_payloads is enabled.
Separate store initialization from the ERA consumer since it's not part of the production beacon node path.
`hot_storage_strategy` reads `hot_hdiff_start_slot()` which returns `anchor_slot`. With the anchor still at slot 0 from `init_genesis_store`, storing the head state at the ERA boundary produces `DiffFrom(intermediate)` and fails because the hot DB has no preceding diffs (we just imported into the cold DB). Update the anchor in memory first so the strategy sees `anchor_slot == head_slot` and stores the head as a `Snapshot`. The kv-op is still added to the same atomic batch as `set_split` and the persisted fork choice for crash-consistent persistence.
Two related fixes for the genesis store init: 1. Store genesis state in the hot DB via `put_state` in addition to the existing `put_cold_state`. The hot state summary is required by `HotColdDB::load_split`, which resolves `split.block_root` via summary on subsequent reopens — needed for crash-resume. 2. Initialize the anchor in memory before `put_state`. `put_state` calls `hot_storage_strategy`, which reads `anchor_slot`; without an initialized anchor it errors with `AnchorUninitialized`. The anchor's kv-op is still included in the same atomic batch with `set_split` and the persisted fork choice for crash-consistent persistence. `put_cold_state` is retained because era 1's `DiffFrom` strategy chains against the slot-0 snapshot in the cold DB (without it, `put_cold_state` in the first era errors with `MissingSnapshot(Slot(0))`).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
ERA file import/export for Lighthouse. Format spec: https://github.com/status-im/nimbus-eth2/blob/stable/docs/the_auditors_handbook/src/02.4_the_era_file_format.md.
Implemented as lcli subcommands so the beacon startup path stays untouched; once exercised this can graduate into beacon. Depends on the reconstruct_historic_states_on_range from #9222.
Import a directory of ERA files into a fresh datadir
Safety checks (ERA data integrity)
against genesis_validators_root.
(post-Capella) of the reference state.
Produce ERA files from a fully reconstructed archive node
After import, store metadata (split, anchor, fork choice) is advanced so the node boots via the regular resume_from_db path.
TODO tests