You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Prater genesis has shown that we can get seriously overloaded with gossip validation work. Where our node spends 15 seconds doing nothing else than validating aggregates
Goals
For our node to survive:
We can't do too much of 1 thing, each module has to yield fast to the macro queue and let other things happen.
Proposal
Add one or multiple validation queue(s) per gossip topic validation.
Ensure that only one aggregate validation is performed at once, instead of 400 as Prater genesis showed.
Add some mechanism for the validation function to yield to the macro queue often. To break long promise chains, we can run a sleep(0) every N jobs.
Limit the queue size and drop jobs after the limit to contain the total load.
Consider doing the queue a LIFO (Last In, First Out). The more recent aggregates are the most valuable so we should prioritize those.
Similarly, consider adding a queue to AttestationProcessor. Then the regen queue may not be needed anymore. Otherwise consider splitting the regen queue into different based on regen method.
The text was updated successfully, but these errors were encountered:
Prater genesis has shown that we can get seriously overloaded with gossip validation work. Where our node spends 15 seconds doing nothing else than validating aggregates
Goals
For our node to survive:
Proposal
Add one or multiple validation queue(s) per gossip topic validation.
sleep(0)
every N jobs.This queue should be implemented in Lodestar side, not gossipsub's. However consider extending this method in our gossipsub implementation so each message is processed serially
https://github.com/libp2p/js-libp2p-interfaces/blob/master/src/pubsub/index.js#L392-L399
Similarly, consider adding a queue to AttestationProcessor. Then the regen queue may not be needed anymore. Otherwise consider splitting the regen queue into different based on regen method.
The text was updated successfully, but these errors were encountered: