Add coarse grained events extension #754
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds the coarse-grained event extension. Similarly to the DPC++ discard events extension, it addresses the performance impact of having to create an event for every single SYCL operation due to the
submit()
signature. However, unlike the DPC++ extension, the events returned with this extension remain valid and support the same functionality as regular events.Instead, the idea is that, when the user is not interested in using the events to construct efficient task graphs, we can just implement the event functionality by synchronizing with the entire backend queue (e.g. CUDA or HIP streams) for backends that are based on in-order queues. Consequently, a coarse-grained event can be implemented without having to construct any backend event. Basically, a coarse-grained event trades worse synchronization performance for better event construction performance/kernel launch latency.
They can either be enabled either only selectively for individual submissions to a queue using a command group property, or for all submissions of a queue using a queue property.
From the documentation:
HIPSYCL_EXT_COARSE_GRAINED_EVENTS
This extension allows to hint to hipSYCL that events associated with command groups can be more coarse-grained and are allowed to synchronize with potentially more operations.
This can allow hipSYCL to trade less synchronization performance for lighter-weight events, and hence lower kernel launch latency. The main benefit are situations where the returned event from
submit
is not of particular interest, e.g. in in-order queues when no user synchronization with those events are expected.For example, a coarse grained event for a backend based on in-order queues (e.g. CUDA or HIP) might just synchronize with the entire HIP or CUDA stream - thereby completely eliding the need to create a new backend event.
Coarse-grained events support the same functionality as regular events.
Coarse-grained events can be requested in two ways:
queue
which instructs thequeue
to construct coarse-grained events for all operations that it processes, andHIPSYCL_EXT_CG_PROPERTY_*
). In this case, coarse-grained events can be enabled selectively only for some command groups submitted to a queue.API Reference
CC @al42and @pszi1ard