Rewrite docs so that it is OK to use record_stream before uses #113282
Conversation
The previous documentation did not appear to accurately describe
the actual semantics in CUDA caching allocator.
When you record stream, we only record a stream use:
```
void recordStream(Block* block, cuda::CUDAStream stream) {
std::lock_guard<std::recursive_mutex> lock(mutex);
if (stream.stream() == block->stream) {
// ignore uses on the allocation stream, since those don't require any
// special synchronization
return;
}
block->stream_uses.insert(stream);
}
```
It is only at deallocation time when we actually install an event on
stream uses that we will subsequently query to determine if the block
can be reused or not.
Signed-off-by: Edward Z. Yang <ezyang@meta.com>
[ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/113282
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (1 Unrelated Failure)As of commit 958446d with merge base 9e6e958 ( UNSTABLE - The following job failed but was likely due to flakiness present on trunk and has been marked as unstable:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
The previous documentation did not appear to accurately describe
the actual semantics in CUDA caching allocator.
When you record stream, we only record a stream use:
```
void recordStream(Block* block, cuda::CUDAStream stream) {
std::lock_guard<std::recursive_mutex> lock(mutex);
if (stream.stream() == block->stream) {
// ignore uses on the allocation stream, since those don't require any
// special synchronization
return;
}
block->stream_uses.insert(stream);
}
```
It is only at deallocation time when we actually install an event on
stream uses that we will subsequently query to determine if the block
can be reused or not.
Signed-off-by: Edward Z. Yang <ezyangmeta.com>
ghstack-source-id: 46c3c2c
Pull Request resolved: #113282
github-actions
bot
requested review from
SherlockNoMad,
albanD,
antoniojkim,
bdhirsh,
miladm,
voznesenskym and
wconstab
|
@pytorchbot merge |
Merge failedReason: This PR needs a If not, please add the To add a label, you can comment to pytorchbot, for example For more information, see Details for Dev Infra teamRaised by workflow job |
ezyang
added
release notes: cuda
|
@pytorchbot merge -f "only the doc job matters" |
Merge startedYour change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
…ch#113282) The previous documentation did not appear to accurately describe the actual semantics in CUDA caching allocator. When you record stream, we only record a stream use: ``` void recordStream(Block* block, cuda::CUDAStream stream) { std::lock_guard<std::recursive_mutex> lock(mutex); if (stream.stream() == block->stream) { // ignore uses on the allocation stream, since those don't require any // special synchronization return; } block->stream_uses.insert(stream); } ``` It is only at deallocation time when we actually install an event on stream uses that we will subsequently query to determine if the block can be reused or not. Signed-off-by: Edward Z. Yang <ezyang@meta.com> Pull Request resolved: pytorch#113282 Approved by: https://github.com/Skylion007, https://github.com/albanD
4 participants
Stack from ghstack (oldest at bottom):
The previous documentation did not appear to accurately describe
the actual semantics in CUDA caching allocator.
When you record stream, we only record a stream use:
It is only at deallocation time when we actually install an event on
stream uses that we will subsequently query to determine if the block
can be reused or not.
Signed-off-by: Edward Z. Yang ezyang@meta.com