-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for generating multiple block proofs at once #296
Comments
@vladimir-trifonov Yeah you're right. Like I guess that PR focuses more on caching block hashes, but this one is more about explicit multi-blocks. |
Ok... So just want to note that while #297 does add support to generate multiple blocks at once, it does not generate multiple blocks in parallel, so this is still something that we should do (as a separate PR probably). |
When we prove a range, let's say blocks 10..13= for processing block 10 we use checkpoint-block-number set to 9, but for processing block 11 we use the block proof of block 10 as a parent, i.e. every next proof relies on the previous one, i.e. that's why proving a range happens synchronously. We can skip that, we have in the readme:
and then the proving blocks could be made to run in parallel... @Nashtare |
I'm not sure how much work this would be with the current zero-bin (cc @cpubot) as it is currently built around proving one block at a time, but we need to be able to process blocks concurrently and to connect them if required, by passing the previous block proof when proving the next one. 99.99% of the proving time (all txn + aggregation proofs) remains fully independent of the previous block, so we could still have the leader push a next block to the queue, to be processed ASAP by available workers.
|
That sounds fine to me, should we make some plan, it sounds like a pretty big change to me if I'm not mistaken... |
I remember I did take a decent look at what an implementation might look like a while back, and IIRC it looked pretty doable with how Paladin was setup. I would take a look at |
Done in 0xPolygonZero/zero-bin#90 |
It has come up during benchmarking that it would be useful to test what it is like to generate multiple block proofs simultaneously.
The motivation for this comes from the fact that the amount of parallelizable work diminishes as generation progresses for a block proof. The reason for this is due to the trie structure from aggregating txn proofs together into a final block proof, the amount of work units available quickly decreases as we move up the proof aggregation trie. This quickly leads to a situation where CPU utilization drops off after a good portion of the proof tree becomes completed.
Because it's useful to benchmark a more realistic scenario where we have multiple blocks being proved at once, we probably should add a new mode that support generating a range of blocks. This should be pretty straight forward to implement I think.
The text was updated successfully, but these errors were encountered: