feat: holesky-resuce - get persisted checkpoint state api#7541
Merged
Conversation
nflaig
reviewed
Mar 10, 2025
| protected readonly blockProcessor: BlockProcessor; | ||
| protected readonly db: IBeaconDb; | ||
| private readonly archiver: Archiver; | ||
| // this is only available if nHistoricalStates is enabled |
Member
There was a problem hiding this comment.
is there any reason we still wanna keep the in-memory state management?
Member
Author
There was a problem hiding this comment.
ah good question, I don't see any benefit to use it from our side
if any consumers want to use proof api, it's not always available with our persistent checkpoint state cache. But we haven't received any complaints since we switch to this cache
so I guess we can remove it at some points to simplify our source code
nflaig
added a commit
that referenced
this pull request
Mar 10, 2025
…ied (#7542) We have added a new api in #7541 to serve latest checkpoint states. However the problem is that `--checkpointState` right now assumes that we receive a ssz-serialized state bytes, so we also need to ensure that the REST API only returns SSZ responses by applying the correct Accept header. This PR includes the following changes - Ensure we only receive SSZ responses if REST API is queried - Consider `--forceCheckpointSync` flag and if set always prioritze remote checkpoint from `--checkpointState` over latest safe checkpoint in db - Minor tweaks to logs to make it easier to understand what the node is doing - Misc formatting changes
nflaig
added a commit
that referenced
this pull request
Oct 1, 2025
**Motivation** - implement an api to get a node synced asap **Description** - new api: `eth/v1/lodestar/persisted_checkpoint_state` to return a state based on an optional `rootHex:epoch` param - if not specified, return the latest safe checkpoint state - a node need to specify `--checkpointState` from the previous PR #7509 **Test** - [x] `curl -H "Accept: application/octet-stream" http://localhost:9596/eth/v1/lodestar/persisted_checkpoint_state -o latest_checkpoint_state.ssz` - [x] `curl -H "Accept: application/octet-stream" http://localhost:9596/eth/v1/lodestar/persisted_checkpoint_state?checkpoint_id=0x4f4d4c1b81141fe77a4a1c6a376dbe64ed9baa8f123664195e4f710c9fc4238d:118936 -o state_epoch_118936.ssz` --------- Co-authored-by: Tuyen Nguyen <twoeths@users.noreply.github.com> Co-authored-by: Nico Flaig <nflaig@protonmail.com>
nflaig
added a commit
that referenced
this pull request
Oct 1, 2025
…ied (#7542) We have added a new api in #7541 to serve latest checkpoint states. However the problem is that `--checkpointState` right now assumes that we receive a ssz-serialized state bytes, so we also need to ensure that the REST API only returns SSZ responses by applying the correct Accept header. This PR includes the following changes - Ensure we only receive SSZ responses if REST API is queried - Consider `--forceCheckpointSync` flag and if set always prioritze remote checkpoint from `--checkpointState` over latest safe checkpoint in db - Minor tweaks to logs to make it easier to understand what the node is doing - Misc formatting changes
wemeetagain
pushed a commit
that referenced
this pull request
Oct 22, 2025
**Motivation** This was a feature we developed for [rescuing Holesky](https://blog.chainsafe.io/lodestar-holesky-rescue-retrospective/) as part of #7501 to quickly sync nodes to head during a period of long non-finality (~3 weeks). While it's unlikely we will have such a long period of non-finality on mainnet, this feature is still useful to have for much shorter periods and testing purposes on devnets. It is now part of [Ethereum protocol hardening](https://github.com/eth-clients/diamond) mitigations described [here](https://github.com/eth-clients/diamond/blob/main/mitigations/nfin-checkpoint-001.md) > Ordinary checkpoint sync begins from the latest finalized checkpoint (block and/or state). As an escape hatch during non-finality, it is useful to have the ability to checkpoint sync from an unfinalized checkpoint. A client implementing this mitigation MUST support checkpoint sync from an arbitrary non-finalized checkpoint state. We will support this with the exception that our checkpoint state needs to be an epoch boundary checkpoint. **Description** The main feature of this PR is to allow initializing a node from an unfinalized checkpoint state either retrieved locally or from a remote source. This behavior is disabled by default but can be enabled by either adding - the `--lastPersistedCheckpointState` flag to load from the last safe persisted checkpoint state stored locally - or `--unsafeCheckpointState` to provide a file path or url to an unfinalized checkpoint state to start syncing from which can be used with new endpoint `GET /eth/v1/lodestar/persisted_checkpoint_state` to sync from a remote node or by sharing states from `checkpoint_states` folder Both of these options are not safe to use on a network that recently finalized an epoch and must only be considered if syncing from last finalized checkpoint state is unfeasible. An unfinalized checkpoint state persisted locally is only considered to be safe to boot if - it's the only checkpoint in it's epoch to avoid ambiguity from forks - its last processed block slot is at an epoch boundary or last slot of previous epoch - state slot is at an epoch boundary - state slot is equal to `epoch * SLOTS_PER_EPOCH` But even if these criteria are met, there is chance that the node will end up on a minority chain as it will not be able to pivot to another chain that conflicts with the checkpoint state it was initialized from. Other existing flags (like `--checkpointState`) are unchanged by this PR and will continue to expect a finalized checkpoint state. Previous PRs #7509, #7541, #7542 not merged to unstable are included. Closes #7963 cc @twoeths --------- Co-authored-by: twoeths <10568965+twoeths@users.noreply.github.com> Co-authored-by: Tuyen Nguyen <twoeths@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Description
eth/v1/lodestar/persisted_checkpoint_stateto return a state based on an optionalrootHex:epochparam--checkpointStatefrom the previous PR feat: use local checkpoint state for holesky-rescue #7509Test
curl -H "Accept: application/octet-stream" http://localhost:9596/eth/v1/lodestar/persisted_checkpoint_state -o latest_checkpoint_state.sszcurl -H "Accept: application/octet-stream" http://localhost:9596/eth/v1/lodestar/persisted_checkpoint_state?checkpoint_id=0x4f4d4c1b81141fe77a4a1c6a376dbe64ed9baa8f123664195e4f710c9fc4238d:118936 -o state_epoch_118936.ssz