Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the sender parallel #70

Merged
merged 13 commits into from
Jun 20, 2023
Merged

Make the sender parallel #70

merged 13 commits into from
Jun 20, 2023

Conversation

bredamatt
Copy link
Contributor

@bredamatt bredamatt commented Jun 20, 2023

Closes #71

This PR makes the sender support parallel execution. Normal usage is still supported, however.

Parallelism is supported by leveraging the same logic, but by replacing the sender_index with a thread index, given that the number of threads passed as an argument to the sender is > 1.

Because there was a mix of I/O bound and CPU bound tasks related to sending transactions using subxt, I split the CPU bound task into a spawn_blocking section, and used a JoinSet for the pre-checks which are I/O bound.
All the transactions are submitted after all them are signed.

There also had to be changes made to the subxt client (Api type here), which was done using the OnlineClient<PolkadotConfig>::from_rpc() method, taking an Arc<RpcClient> as an argument. This is because the default OnlineClient does not support the number of concurrent requests required when the sender in parallel submits all the transactions. Therefore, the client has to be built with the WsClientBuilder from jsonrpsee to set the max_concurrent_requests parameter in stead. Currently, the max_concurrent_requests is set to u32::MAX.

@bredamatt bredamatt self-assigned this Jun 20, 2023
@bredamatt bredamatt added the enhancement New feature or request label Jun 20, 2023
@bredamatt bredamatt requested a review from pepoviola June 20, 2023 13:16
utils/sender/src/main.rs Outdated Show resolved Hide resolved
utils/sender/src/main.rs Outdated Show resolved Hide resolved
@bredamatt bredamatt requested a review from pepoviola June 20, 2023 15:05
@bredamatt
Copy link
Contributor Author

Thanks for the comments @pepoviola ! All of them should be addressed by now.

@bredamatt bredamatt merged commit 3fc1ef4 into main Jun 20, 2023
2 checks passed
@bredamatt bredamatt deleted the bredamatt/sender-parallelism branch June 20, 2023 17:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
No open projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Make the sender parallel
2 participants