Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Runtime Environment] Parse special characters in private Git URIs #28250

Merged
merged 2 commits into from Sep 2, 2022

Conversation

shrekris-anyscale
Copy link
Contributor

Why are these changes needed?

Users can access private Git repos via runtime environments by specifying an archive link, usually containing a username and private token. For example, from the Ray documentation, the GitHub format is

https://[username]:[personal access token]@github.com/[username]/[private repository]/archive/[commit hash].zip

However, the colon (:) and at (@) symbols are not removed before parsing the URI as a directory name for the runtime environment. Since PYTHONPATHs use colons as package name separators, Python modules in this private repo can't be imported.

This change converts the colon and at symbols to underscores (_) when converting the URI into a directory name. This allows modules stored in private repos to be imported.

Related issue number

N/A

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
      • This change adds new unit tests to test_runtime_env_packaging.py.

Signed-off-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Signed-off-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Copy link
Contributor

@simon-mo simon-mo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM as long as manually tested as well

@shrekris-anyscale
Copy link
Contributor Author

I just verified manually by testing that imports from a private repo work.

@simon-mo simon-mo merged commit 3b7346a into ray-project:master Sep 2, 2022
kira-lin pushed a commit to kira-lin/ray that referenced this pull request Sep 8, 2022
ilee300a pushed a commit to ilee300a/ray that referenced this pull request Sep 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants