ERA file support

[philknows]

Problem description

As Ethereum moves towards a world where historical information over 1 year is purged on nodes that don't really need it, support has been gaining around ERA files .e2s to store and redistribute execution, but also beacon chain history and state. We should be able to support integrating this data structure format in Lodestar.

Solution description

Copying (modified) from discord https://discord.com/channels/593655374469660673/1336849245281587321/1349019517812215890:

• Use ERA files for historical state regen (as an alternative to db-managed backfill/sync from genesis)
  something like --era-dir flag, which gets fed into the chain and used in our historical state regen worker . The behavior being something like:

  • check for an ERA file for the requested range
  • check the db for a prestate/blocks
  • pick the best option for regen

• Use ERA files to serve beacon_blocks_by_range

• Produce ERA files, behavior being something like:

  • on finalization, check if new ERAs can be produced
  • produce ERA file
  • this could be done in a worker

Additional context

Implementation of e2store files by Nimbus: https://github.com/status-im/nimbus-eth2/blob/stable/docs/e2store.md
EthResearch Post: https://ethresear.ch/t/era-archival-files-for-block-and-consensus-data/13526
Ultralight (Portal Network) integration: ethereumjs/ultralight#613

[philknows]

Requirements (do we want to do these?):

• Tool to create an .era file
• Being able to configure the BN to point to a dir with the era files and use them for historical state type tasks.
• There are multiple points in the BN to regenerate the data files: Once every period after the period finalizes. Or after X months (after the block data availability period is over during prune).

[acolytec3]

FYI, we've built out era support within the ethjs monorepo. It's not published yet but should be with our next round of releases (which hopefully happens sooner rather than later).

Note, for the original era format, we don't do all of the validation specified by the era specs but this is could be done using the Beacon State snapshot and the blocks produced by readBlocksFromEra function.

[philknows]

Note here that we made a decision on July 8 to take into consideration ERB support when the specification is finalized also. This issue should take into consideration that this is anticipated and package structuring should consolidate all E2Store functionality into a single package. See: https://github.com/ChainSafe/lodestar/wiki/Lodestar-Weekly-Standups-2025#era-format-for-blobs

[acolytec3]

For future reference, @ScottyPoi and I finished out support for the various e2store formats we were aware of here so this could be leveraged as prior art whenever you move forward on a unified e2store package (provided you're implementing in typescript).