Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: use --syncmode=execution-layer from op-node for optimistic pipeline sync #7552

Merged
merged 62 commits into from
May 7, 2024

Conversation

joshieDo
Copy link
Collaborator

@joshieDo joshieDo commented Apr 10, 2024

Adds optimistic sync (unsafe head hash & finalized zero hash). Although a requirement for optimism, it's also an expectation in non OP chains.

  • Passes an unfinalized hash as target to pipeline
  • handles reorgs from optimistic targets, by issuing an unwind from BT to pipeline: PipelineTarget::Unwind(block)
  • crates/optimism/node/tests/e2e/p2p.rs has an example scenario

Pending: syncing base main from scratch.

--

OP specific changes:
https://github.com/paradigmxyz/reth/pull/7552/files#diff-c821f432ea2a3ba2fdad959da77aae18c2b20e4988e032aa856576cfafb3a58a: op-node will ignore --syncmode=execution-layer, if the EL has any finalized block. This makes sure that we don't start with one (as is usual behaviour). Otherwise, op-node will move on to a CL sync style, feeding us one block at a time from our lowest.

The error from UnknownSafeOrFinalizedBlock has to start with an upper case since it's what op-node requires: https://github.com/ethereum-optimism/optimism/blob/0913776869f6cb2c1218497463d7377cf4de16de/op-service/sources/l2_client.go#L105. This is temporary until a version of the following PR is pushed through, where the tracking of the EL status is properly tracked.

@joshieDo joshieDo changed the title Joshie/hot op fix: op-reth now uses --syncmode=execution-layer from op-node on starting up Apr 10, 2024
@joshieDo joshieDo added C-bug An unexpected or incorrect behavior A-op-reth Related to Optimism and op-reth labels Apr 10, 2024
@onbjerg
Copy link
Member

onbjerg commented Apr 10, 2024

smol meta nit: can we make the PR title imperative 😄

@joshieDo joshieDo changed the title fix: op-reth now uses --syncmode=execution-layer from op-node on starting up fix: use --syncmode=execution-layer from op-node for pipeline sync Apr 10, 2024
crates/consensus/beacon/src/engine/mod.rs Outdated Show resolved Hide resolved
/// op-node uses case sensitive string comparison to parse this error:
/// https://github.com/ethereum-optimism/optimism/blob/0913776869f6cb2c1218497463d7377cf4de16de/op-service/sources/l2_client.go#L105
///
/// Temporary, until a version of https://github.com/ethereum-optimism/optimism/pull/10071 is pushed through that doesn't require this to figure out the EL sync status.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's also track this

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +1763 to +1766
} else if error.optimistic_revert_block_number().is_some() {
// engine already set the pipeline unwind target on
// `try_make_sync_target_canonical`
PayloadStatus::from_status(PayloadStatusEnum::Syncing)
Copy link
Collaborator Author

@joshieDo joshieDo May 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unsure here if returning Syncing is the best

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be okay

@@ -668,6 +669,21 @@ where
// threshold
return Some(state.finalized_block_hash)
}

// OPTIMISTIC SYNCING
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we feature gate this?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or should we do this only for OP (chainspec.is_optimism) for now?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i mean, its unlikely that it can happen on mainnet, but it can happen on testnets/devnets l1 for sure.

i can add the feature gate or check... but what would be the reason?

@joshieDo
Copy link
Collaborator Author

joshieDo commented May 3, 2024

1st run of hive tests seemed to fail 2 tests,

tried locally and they were all passing so issued a second action and they all passed same as main: https://github.com/paradigmxyz/reth/actions/runs/8930154135

@joshieDo
Copy link
Collaborator Author

joshieDo commented May 3, 2024

eth-holesky: triggered optimistic sync on lighthouse + reth (although low block, instead of near the tip)

Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

few more doc nits, but overall I think all of this is fine

Comment on lines -57 to -59
head_block_hash: head.hash(),
safe_block_hash: head.hash(),
finalized_block_hash: head.hash(),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure this was problematic from the start

crates/blockchain-tree/src/blockchain_tree.rs Show resolved Hide resolved
crates/consensus/beacon/src/engine/mod.rs Outdated Show resolved Hide resolved
Comment on lines +1763 to +1766
} else if error.optimistic_revert_block_number().is_some() {
// engine already set the pipeline unwind target on
// `try_make_sync_target_canonical`
PayloadStatus::from_status(PayloadStatusEnum::Syncing)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be okay

crates/primitives/src/stage/mod.rs Outdated Show resolved Hide resolved
crates/stages-api/src/pipeline/mod.rs Outdated Show resolved Hide resolved
@@ -668,6 +669,21 @@ where
// threshold
return Some(state.finalized_block_hash)
}

// OPTIMISTIC SYNCING
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or should we do this only for OP (chainspec.is_optimism) for now?

Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pending @rkrasiuk

@gakonst gakonst added this pull request to the merge queue May 7, 2024
Merged via the queue into main with commit 9bd74fd May 7, 2024
28 checks passed
@gakonst gakonst deleted the joshie/hot-op branch May 7, 2024 20:29
mw2000 pushed a commit to mw2000/reth that referenced this pull request Jun 5, 2024
…ipeline sync (paradigmxyz#7552)

Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-op-reth Related to Optimism and op-reth C-bug An unexpected or incorrect behavior
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants