Skip to content

Return device count using nvml #84879

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 4 commits into from
Closed

Return device count using nvml #84879

wants to merge 4 commits into from

Conversation

malfet
Copy link
Contributor

@malfet malfet commented Sep 12, 2022 •
edited
Loading

@malfet
Return device count using nvml
702e1e0 
Fixes #83973

[ghstack-poisoned]
This was referenced Sep 12, 2022
Closed
Closed
@pytorch-bot
Copy link

pytorch-bot bot commented Sep 12, 2022 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/84879

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit a5ebf40:
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@malfet malfet requested a review from ngimel September 12, 2022 18:28
malfet added a commit that referenced this pull request Sep 12, 2022
@malfet
Return device count using nvml
a776ebe 
Fixes #83973

ghstack-source-id: 59c92e0
Pull Request resolved: #84879
ngimel
ngimel approved these changes Sep 12, 2022
View reviewed changes
torch/cuda/__init__.py
return int(s[:idx]) if idx > 0 else -1

# CUDA_VISIBLE_DEVICES uses something like strtoul
# which makes `1gpu2,2ampere` is equivalent to `1,2`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

til

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't handle things like 0-2?

Copy link
Contributor Author

@malfet malfet Sep 12, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, at using 510 driver, ranges do not seem to be supported (prior to this change):

$ CUDA_VISIBLE_DEVICES=0-2 python -c "import torch;print(torch.cuda.device_count())"
1
$ CUDA_VISIBLE_DEVICES=0,1,2 python -c "import torch;print(torch.cuda.device_count())"
3

malfet added a commit that referenced this pull request Sep 13, 2022
@malfet
Return device count using nvml
0a74b81 
Fixes #83973

ghstack-source-id: afb86e6
Pull Request resolved: #84879
@malfet malfet added the ciflow/trunk Trigger trunk jobs on your pull request label Sep 13, 2022
@malfet
Copy link
Contributor Author

malfet commented Sep 13, 2022

@pytorchbot merge -g

@malfet malfet added release notes: cuda release notes category topic: improvements topic category labels Sep 13, 2022
@pytorchmergebot
Copy link
Collaborator

@pytorchbot successfully started a merge job. Check the current status here.
The merge job was triggered with the green (-g) flag. This means that your change will be merged once all checks on your PR have passed (ETA: 0-4 Hours). If this is not the intended behavior, feel free to use some of the other merge options in the wiki.
Please reach out to the PyTorch DevX Team with feedback or questions!

This was referenced Sep 14, 2022
Closed
Merged
Merged
@facebook-github-bot facebook-github-bot deleted the gh/malfet/48/head branch September 17, 2022 14:19
mehtanirav pushed a commit that referenced this pull request Oct 4, 2022
@malfet @mehtanirav
Return device count using nvml (#84879)
8827cd1 
Fixes #83973
Pull Request resolved: #84879
Approved by: https://github.com/ngimel
@vadimcn vadimcn mentioned this pull request Dec 17, 2022
Closed
@awaelchli awaelchli mentioned this pull request Feb 6, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ngimel ngimel ngimel approved these changes

Assignees
No one assigned
Labels
ciflow/trunk Trigger trunk jobs on your pull request cla signed Merged release notes: cuda release notes category topic: improvements topic category
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@malfet @pytorchmergebot @ngimel @facebook-github-bot