Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Optimize] Add guards for processing unconfirmed solutions #3225

Merged
merged 8 commits into from Apr 24, 2024

Conversation

raychu86
Copy link
Contributor

Motivation

This PR adds a few changes to improve syncing and reduce network spam:

  1. Clients perform solution verification in POST /broadcast/solution prior to solution propagation.
    • This reduces the propagation of invalid solutions on the network.
  2. Nodes do not process incoming unconfirmed solutions if they are syncing.
    • This reduces the resource load of nodes and lets them focus on syncing.
  3. Nodes do not process incoming unconfirmed transactions if they are syncing
    • This reduces the resource load of nodes and lets them focus on syncing.

@@ -338,9 +338,27 @@ impl<N: Network, C: ConsensusStorage<N>, R: Routing<N>> Rest<N, C, R> {
Json(solution): Json<Solution<N>>,
) -> Result<ErasedJson, RestError> {
// If the consensus module is enabled, add the unconfirmed solution to the memory pool.
if let Some(consensus) = rest.consensus {
// Otherwise, verify it prior to broadcasting.
Copy link
Collaborator

@vicsn vicsn Apr 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can imagine spamming a publicly accessible REST endpoint with solutions could halt a client due to the amount of verifications required. But we should test (cc @iamalwaysuncomfortable).

If it turns out to be an issue, we can:

  • accept the risk and tell everyone not to expose their REST endpoint...
  • add a separate transmission rate limit using a second .layer(GovernorLayer { }.
  • queue transmissions to verify, takes more work but seems more robust. See my comment below for an example implementation

consensus.add_unconfirmed_solution(solution).await?;
Some(consensus) => consensus.add_unconfirmed_solution(solution).await?,
// Verify the solution.
None => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If clients receive transactions/solutions via P2P, they are already verified before propagation in fn unconfirmed_{transaction, solution}. So I suggest we just call that function from here. This is what the validator also does.

For consistency and clarity, we should also do this for transactions. A solution takes longer to verify than an execution, but less than a deployment, so there's no difference justifying a different verification approach.

Recall this PR was triggered in part because recently we did a large invalid solution spam, but @iamalwaysuncomfortable and I just realized we never tried spamming very large amounts of invalid transactions.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR also remains relevant in the face of solution or deployment spam: #2970

@iamalwaysuncomfortable
Copy link
Contributor

iamalwaysuncomfortable commented Apr 23, 2024

The following measures are being done to test this

  • Run a 30 client, 10 validator devnet with a high load of solution spam
  • Analyze the results

The devnet run has completed, and analysis of it is forthcoming.

@howardwu howardwu merged commit a83a767 into mainnet-staging Apr 24, 2024
18 of 24 checks passed
@howardwu howardwu deleted the optimize/solution-processing branch April 24, 2024 22:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants