Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove \ in cache_dir #110945

Closed

Conversation

@pytorch-bot
Copy link

pytorch-bot bot commented Oct 10, 2023

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/110945

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 35719b6 with merge base 2bc1378 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@taomiao
Copy link
Contributor Author

taomiao commented Oct 10, 2023

@pytorchbot label "topic: not user facing"

@masnesral
Copy link
Contributor

@taomiao can you explain where the backslash is coming from? Is it for win, for example? If so, should we be using os.path.join to form the path?

@taomiao taomiao force-pushed the FIX/cache_dir_contains_backslash branch 3 times, most recently from 81dafde to cfe7872 Compare October 11, 2023 02:38
@taomiao
Copy link
Contributor Author

taomiao commented Oct 11, 2023

@masnesral getpass.getuser() return username like "ORG\user1", This kind of username is more common among institutional users. This happens in both windows and linux.

@cpuhrsch cpuhrsch added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Oct 11, 2023
@masnesral
Copy link
Contributor

@masnesral getpass.getuser() return username like "ORG\user1", This kind of username is more common among institutional users. This happens in both windows and linux.

Oh, ok. Thanks for explaining. In that case, probably we should apply the \ replacement only to the user string portion. And since you're changing this code, would you mind switching it to use os.path.join rather that hard-coding the front slash?

torch/_inductor/codecache.py Outdated Show resolved Hide resolved
@taomiao taomiao force-pushed the FIX/cache_dir_contains_backslash branch from cfe7872 to 4d91f61 Compare October 12, 2023 02:41
Copy link
Contributor

@masnesral masnesral left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: The formatting looks weird; perhaps you can fix before landing (I would have expected the linter to fix). But the logic looks good otherwise. Thanks for the changes.

@taomiao taomiao force-pushed the FIX/cache_dir_contains_backslash branch 2 times, most recently from 7cebbf2 to 61b08d0 Compare October 12, 2023 04:40
@taomiao
Copy link
Contributor Author

taomiao commented Oct 12, 2023

nit: The formatting looks weird; perhaps you can fix before landing (I would have expected the linter to fix). But the logic looks good otherwise. Thanks for the changes.

ok

@taomiao
Copy link
Contributor Author

taomiao commented Oct 16, 2023

@shunting314 Do you have a minute for this?

Copy link
Contributor

@shunting314 shunting314 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG!

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Oct 17, 2023
@pytorchmergebot
Copy link
Collaborator

Merge failed

Reason: Comment with id 1765974981 not found

Details for Dev Infra team Raised by workflow job

@taomiao
Copy link
Contributor Author

taomiao commented Oct 17, 2023

@cpuhrsch ok to be merged.

@kit1980
Copy link
Member

kit1980 commented Oct 26, 2023

@taomiao FYI when the PR is approved by a team member, you can merge yourself by posting "@pytorchbot merge" comment.

Copy link
Member

@kit1980 kit1980 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I agree with this PR.
Do we really care about "ORG\user1" usernames? Then what about other symbols that can be in usernames but not in paths?
Also the current PR will map "ORG\user1" and "ORGuser1" to the same dir.

So I'm not sure if we really care about this use case.
And if we do care, this needs to be done in a more generic and robust way.

@masnesral
Copy link
Contributor

And if we do care, this needs to be done in a more generic and robust way.

I'm not sure if there's a canonical way to sanitize the string to make it appropriate for paths. Maybe something like this is a bit more generic:

    illegal_chars = r'[\/:*?"<>|]'
    sanitized_name = re.sub(illegal_chars, '_', name)

@taomiao
Copy link
Contributor Author

taomiao commented Oct 30, 2023

I'm not sure I agree with this PR. Do we really care about "ORG\user1" usernames? Then what about other symbols that can be in usernames but not in paths? Also the current PR will map "ORG\user1" and "ORGuser1" to the same dir.

So I'm not sure if we really care about this use case. And if we do care, this needs to be done in a more generic and robust way.

Some usernames do cause inductor's cache directory path "can not be found". This kind of error will make uninformed users feel baffled. Sure if there's a canonical way to sanitize the string to make it appropriate for paths would be better.

@taomiao
Copy link
Contributor Author

taomiao commented Oct 30, 2023

And if we do care, this needs to be done in a more generic and robust way.

I'm not sure if there's a canonical way to sanitize the string to make it appropriate for paths. Maybe something like this is a bit more generic:

    illegal_chars = r'[\/:*?"<>|]'
    sanitized_name = re.sub(illegal_chars, '_', name)

Ok

@taomiao taomiao force-pushed the FIX/cache_dir_contains_backslash branch 4 times, most recently from 6e9abd7 to bc7dc2e Compare October 30, 2023 07:17
@taomiao taomiao requested a review from kit1980 October 30, 2023 07:19
@pytorch-bot
Copy link

pytorch-bot bot commented Oct 30, 2023

You don't have permissions to rebase this PR since you are a first time contributor. If you think this is a mistake, please contact PyTorch Dev Infra.

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@pytorchmergebot
Copy link
Collaborator

Merge failed

Reason: 2 mandatory check(s) failed. The first few are:

Dig deeper by viewing the failures on hud

Details for Dev Infra team Raised by workflow job

Failing merge rule: Core Maintainers

@taomiao taomiao force-pushed the FIX/cache_dir_contains_backslash branch from a585e7e to 35719b6 Compare November 6, 2023 06:36
@taomiao
Copy link
Contributor Author

taomiao commented Nov 9, 2023

@kit1980 changed to more generic.

@taomiao
Copy link
Contributor Author

taomiao commented Nov 15, 2023

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ciflow/inductor ciflow/trunk Trigger trunk jobs on your pull request Merged module: inductor open source topic: not user facing topic category triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Development

Successfully merging this pull request may close these issues.

torch.compile failed. "g++: error: /tmp/torchinductor_**** file not found"
7 participants