Collect more info on cuduEventRecord for stream wait sync events #808
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This pull request was exported from Phabricator. Differential Revision: D49694339 |
…orch#808) Summary: ## Overview To effectively understand synchronization due to CUDA events we also need to track CUDA event record calls and which CUDA event record call the sync occurss on. Thus when an event synchronization occurs we track 1. We can figure out which cudaEventRecord() this sync took place on 2. Based on that we know which kernel/memcpy launch was recorded by the event, this will typically be the previous kernel/memcpy launch in runtime. CUPTI' CUDA Event record tells us the correlation ID of the CUDA record [documentation](https://docs.nvidia.com/cupti/annotated.html#structCUpti__ActivityEvent) ``` uint32_t CUpti_ActivityEvent::correlationId [inherited] The correlation ID of the event. Use of this ID is user-defined, but typically this ID value will equal the correlation ID of the kernel for which the event was gathered. ``` ## In this change * Enable logging cuda Event Record Runtime events (and event sychronize calls too). * Track the correlation ID for CUPTI Event records as shown above. * In the CUPTI Stream Wait event we emit the correlation Reviewed By: anupambhatnagar, davidberard98 Differential Revision: D49694339
|
This pull request was exported from Phabricator. Differential Revision: D49694339 |
briancoutinho
force-pushed
the
export-D49694339
branch
from
8a10b4b
to
bbc580d
Compare
|
This pull request was exported from Phabricator. Differential Revision: D49694339 |
|
This pull request has been merged in 4df86b0. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Overview
To effectively understand synchronization due to CUDA events we also need to track CUDA event record calls and which CUDA event record call the sync occurss on.
Thus when an event synchronization occurs we track
CUPTI' CUDA Event record tells us the correlation ID of the CUDA record documentation
In this change
Reviewed By: anupambhatnagar, davidberard98
Differential Revision: D49694339