Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmark for VPIC-style workload #231

Merged
merged 19 commits into from
Oct 1, 2021
Merged

Conversation

ChristopherHogan
Copy link
Collaborator

Adds changes required for getting the best performance from write-only workloads.

  • Control the stdio adapter's page size with the HERMES_PAGE_SIZE environment variable.
  • Add rr_retry to api::Context. This is will retry failed round robin placements with the next Target. It mitigates the problem of several ranks doing a Put where each rank gets a valid schema, but only some of the ranks succeed in getting the buffers. For cases like VPIC where we know we have enough buffering space, we can enable this option to keep trying until GetBuffers succeeds.
  • Added api::Context::disable_swap. Useful in tandom with rr_retry.
  • RoundRobinPlacement now takes a split parameter to explicitly enable or disable splitting instead of leaving to random chance. This can be controlled globally via the default_rr_split config variable, or via the api::Context::rr_split option.
  • Made several API arguments const.
  • Removed RPC versions of ShutdownBufferOrganizer. The BO cannot be shutdown via RPC because the RPC servers need to be shut down before the BO (so that there are no outstanding operations in flight).

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.1%) to 81.19% when pulling 4be7aab on chogan/vpic2 into e10e815 on master.

@ChristopherHogan ChristopherHogan merged commit 8331bda into master Oct 1, 2021
@ChristopherHogan ChristopherHogan deleted the chogan/vpic2 branch October 1, 2021 14:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants