Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CUDA Host Allocator] Add support of CudaHostRegister #108488

Closed
wants to merge 1 commit into from
Closed

[CUDA Host Allocator] Add support of CudaHostRegister #108488

wants to merge 1 commit into from

Conversation

banitag1
Copy link
Contributor

@banitag1 banitag1 commented Sep 3, 2023

Summary: This diff adds another option to create cuda pinned memory using cudaHostRegister.

Differential Revision: D45843715

@pytorch-bot
Copy link

pytorch-bot bot commented Sep 3, 2023 •
edited

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/108488

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (3 Unrelated Failures)

As of commit 7b5470a with merge base a0cea51 (image):

UNSTABLE - The following jobs failed but were likely due to flakiness present on trunk and has been marked as unstable:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D45843715

@github-actions
Copy link

github-actions bot commented Sep 3, 2023

This PR needs a release notes: label

If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

banitag1 added a commit to banitag1/pytorch that referenced this pull request Sep 3, 2023
@banitag1 @facebook-github-bot
[CUDA Host Allocator] Add support of CudaHostRegister (pytorch#108488)
1eeddd4 
Summary:

This diff adds another option to create cuda pinned memory using cudaHostRegister.

Differential Revision: D45843715
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D45843715

banitag1 added a commit to banitag1/pytorch that referenced this pull request Sep 5, 2023
@banitag1 @facebook-github-bot
[CUDA Host Allocator] Add support of CudaHostRegister (pytorch#108488)
ffb60f7 
Summary:

This diff adds another option to create cuda pinned memory using cudaHostRegister.

Differential Revision: D45843715
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D45843715

1 similar comment
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D45843715

banitag1 added a commit to banitag1/pytorch that referenced this pull request Oct 4, 2023
@banitag1 @facebook-github-bot
[CUDA Host Allocator] Add support of CudaHostRegister (pytorch#108488)
415844f 
Summary:

This diff adds another option to create cuda pinned memory using cudaHostRegister to avoid large lock wait time with cudaHostAlloc.

Differential Revision: D45843715
banitag1 added a commit to banitag1/pytorch that referenced this pull request Oct 4, 2023
@banitag1 @facebook-github-bot
[CUDA Host Allocator] Add support of CudaHostRegister (pytorch#108488)
3ace388 
Summary:

This diff adds another option to create cuda pinned memory using cudaHostRegister to avoid large lock wait time with cudaHostAlloc.

Differential Revision: D45843715
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D45843715

banitag1 added a commit to banitag1/pytorch that referenced this pull request Oct 4, 2023
@banitag1 @facebook-github-bot
[CUDA Host Allocator] Add support of CudaHostRegister (pytorch#108488)
e3efe51 
Summary:

This diff adds another option to create cuda pinned memory using cudaHostRegister to avoid large lock wait time with cudaHostAlloc.

Differential Revision: D45843715
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D45843715

banitag1 added a commit to banitag1/pytorch that referenced this pull request Oct 4, 2023
@banitag1 @facebook-github-bot
[CUDA Host Allocator] Add support of CudaHostRegister (pytorch#108488)
f3e7a3d 
Summary:

This diff adds another option to create cuda pinned memory using cudaHostRegister to avoid large lock wait time with cudaHostAlloc.

Differential Revision: D45843715
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D45843715

banitag1 added a commit to banitag1/pytorch that referenced this pull request Oct 4, 2023
@banitag1 @facebook-github-bot
[CUDA Host Allocator] Add support of CudaHostRegister (pytorch#108488)
77048d1 
Summary:

This diff adds another option to create cuda pinned memory using cudaHostRegister to avoid large lock wait time with cudaHostAlloc.

Differential Revision: D45843715
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D45843715

banitag1 added a commit to banitag1/pytorch that referenced this pull request Oct 5, 2023
@banitag1 @facebook-github-bot
[CUDA Host Allocator] Add support of CudaHostRegister (pytorch#108488)
20b7bb6 
Summary:

This diff adds another option to create cuda pinned memory using cudaHostRegister to avoid large lock wait time with cudaHostAlloc.

Differential Revision: D45843715
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D45843715

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D45843715

banitag1 added a commit to banitag1/pytorch that referenced this pull request Oct 5, 2023
@banitag1 @facebook-github-bot
[CUDA Host Allocator] Add support of CudaHostRegister (pytorch#108488)
d20ba0c 
Summary:

This diff adds another option to create cuda pinned memory using cudaHostRegister to avoid large lock wait time with cudaHostAlloc.

Differential Revision: D45843715
@banitag1 banitag1 force-pushed the export-D45843715 branch 2 times, most recently from d20ba0c to 8e49959 Compare October 5, 2023 02:52
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D45843715

banitag1 added a commit to banitag1/pytorch that referenced this pull request Oct 5, 2023
@banitag1 @facebook-github-bot
[CUDA Host Allocator] Add support of CudaHostRegister (pytorch#108488)
8e49959 
Summary:

This diff adds another option to create cuda pinned memory using cudaHostRegister to avoid large lock wait time with cudaHostAlloc.

Differential Revision: D45843715
zdevito
zdevito approved these changes Oct 5, 2023
View reviewed changes
test/test_cuda.py Outdated
@@ -76,6 +76,13 @@ def tearDown(self):
del self.autocast_lists
super().tearDown()

def test_pinned_memory_with_cudaregister(self):
os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "pinned_use_cuda_host_register:True,pinned_num_register_threads:8"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be torch.cuda.memory._set_allocator_settings otherwise it won't have an effect. Make sure to set it back after the test.

banitag1 added a commit to banitag1/pytorch that referenced this pull request Oct 5, 2023
@banitag1 @facebook-github-bot
[CUDA Host Allocator] Add support of CudaHostRegister (pytorch#108488)
b035fb5 
Summary:

This diff adds another option to create cuda pinned memory using cudaHostRegister to avoid large lock wait time with cudaHostAlloc.

Reviewed By: zdevito

Differential Revision: D45843715
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D45843715

@banitag1 @facebook-github-bot
[CUDA Host Allocator] Add support of CudaHostRegister (pytorch#108488)
7b5470a 
Summary:

This diff adds another option to create cuda pinned memory using cudaHostRegister to avoid large lock wait time with cudaHostAlloc.

Reviewed By: zdevito

Differential Revision: D45843715
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D45843715

@facebook-github-bot
Copy link
Contributor

@pytorchbot merge

(Initiating merge automatically since Phabricator Diff has merged)

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Oct 6, 2023
@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@jeffdaily jeffdaily mentioned this pull request Oct 6, 2023
Closed
pytorchmergebot pushed a commit that referenced this pull request Oct 6, 2023
@jeffdaily @pytorchmergebot
[CUDA Host Allocator][ROCm] fixes (#110715)
59592ce 
Follow up to #110123, removing the CUDA_VERSION check for ROCm because HIP already has hipMallocAsync() and doesn't need the version check there.

Follow up to #108488, fixing the unit failing unit tests by accepting either a "cuda" or "hip" attribute for the caching allocator options.  This is aligned to the masquerading strategy for ROCm/HIP.

Pull Request resolved: #110715
Approved by: https://github.com/ezyang
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zdevito zdevito zdevito approved these changes

@jeffdaily jeffdaily Awaiting requested review from jeffdaily jeffdaily is a code owner

@jithunnair-amd jithunnair-amd Awaiting requested review from jithunnair-amd jithunnair-amd is a code owner

Assignees
No one assigned
Labels
ciflow/trunk Trigger trunk jobs on your pull request fb-exported Merged
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@banitag1 @facebook-github-bot @pytorchmergebot @zdevito