Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CUDA][HIP] Add event pool #757

Merged
merged 2 commits into from
Jun 27, 2022
Merged

[CUDA][HIP] Add event pool #757

merged 2 commits into from
Jun 27, 2022

Conversation

illuhad
Copy link
Collaborator

@illuhad illuhad commented Jun 26, 2022

Previously, a new CUDA or HIP event was created for every new operation. Calls to cudaEventCreate() or hipEventCreate can however be costly.

This PR adds an event pool for the CUDA and HIP backends. It works as follows:

  • We maintain a per-device list of events that are currently unused (event pool).
  • When we need a new event, we first check in the event pool whether there are unused events available
    • if so, we pop one from there and use it.
    • If not, we construct a new one using cudaEventCreate()/hipEventCreate().
  • In the event destructor, the backend event is returned to the event pool.

This should allow us to elide most calls to event creation functions.

CC @al42and @pszi1ard

@illuhad illuhad marked this pull request as ready for review June 26, 2022 13:30
@illuhad illuhad merged commit 3da2b40 into develop Jun 27, 2022
@illuhad illuhad deleted the feature/event-pool branch June 27, 2022 18:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant