Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add queue in gossip validation #2265

Closed
dapplion opened this issue Mar 25, 2021 · 0 comments · Fixed by #2277
Closed

Add queue in gossip validation #2265

dapplion opened this issue Mar 25, 2021 · 0 comments · Fixed by #2277
Assignees
Labels
meta-discussion Indicates a topic that requires input from various developers.

Comments

@dapplion
Copy link
Contributor

dapplion commented Mar 25, 2021

Prater genesis has shown that we can get seriously overloaded with gossip validation work. Where our node spends 15 seconds doing nothing else than validating aggregates

Goals
For our node to survive:

  • We can't do too much of 1 thing, each module has to yield fast to the macro queue and let other things happen.

Proposal
Add one or multiple validation queue(s) per gossip topic validation.

  • Ensure that only one aggregate validation is performed at once, instead of 400 as Prater genesis showed.
  • Add some mechanism for the validation function to yield to the macro queue often. To break long promise chains, we can run a sleep(0) every N jobs.
  • Limit the queue size and drop jobs after the limit to contain the total load.
  • Consider doing the queue a LIFO (Last In, First Out). The more recent aggregates are the most valuable so we should prioritize those.

This queue should be implemented in Lodestar side, not gossipsub's. However consider extending this method in our gossipsub implementation so each message is processed serially
https://github.com/libp2p/js-libp2p-interfaces/blob/master/src/pubsub/index.js#L392-L399


Similarly, consider adding a queue to AttestationProcessor. Then the regen queue may not be needed anymore. Otherwise consider splitting the regen queue into different based on regen method.

@dapplion dapplion self-assigned this Mar 25, 2021
@dapplion dapplion added the meta-discussion Indicates a topic that requires input from various developers. label Mar 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
meta-discussion Indicates a topic that requires input from various developers.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant