Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Python command absolute before calling subprocess #300

Merged

Conversation

@uranusjr
Copy link
Contributor

uranusjr commented Nov 27, 2019

bpo-38905

On Python 3.7.2 or later, sys.executable returns the wrong executable inside venv on Windows unless you invoke the interpreter with an absolute path. It seems that the behaviour is expected and unlikely to be fixed (at least not any time soon). Core devs seem to approve the shutil.which() workaround.

This works around bpo-38905 with suggestion from Steve Dower.

https://bugs.python.org/issue38905
@takluyver

This comment has been minimized.

Copy link
Owner

takluyver commented Nov 27, 2019

That's rather frustrating. I've worked on a bunch of code that relies on sys.executable giving me an absolute, reliable path, regardless of how Python was invoked.

If we're using which() to get an absolute path anyway, does then finding sys.executable ever make any difference?

@takluyver

This comment has been minimized.

Copy link
Owner

takluyver commented Nov 27, 2019

I was just about to post a grumpy message to that Python issue, but on reading it more carefully, I think I see it's not sys.executable that is giving you the wrong result, but the subprocess launch that's doing something different from a normal PATH search. That's less concerning; I'm OK with doing a which() on it in Python.

@@ -22,6 +23,7 @@ def find_python_executable(python):
return sys.executable
if os.path.isabs(python): # sys.executable is absolute too
return python
python = shutil.which(python) or os.path.abspath(python)

This comment has been minimized.

Copy link
@takluyver

takluyver Nov 27, 2019

Owner

Why the abspath call? I think which() already handles relative paths.

This comment has been minimized.

Copy link
@uranusjr

uranusjr Nov 27, 2019

Author Contributor

I was not aware of this, thanks! TIL 😄

This comment has been minimized.

Copy link
@uranusjr

uranusjr Nov 27, 2019

Author Contributor

What should we fall back to if the which call fails though? (It returns None if there’s no matching executable found.) Just the unresolved python? Because otherwise subprocess.check_call would fail with a cryptic TypeError.

This comment has been minimized.

Copy link
@takluyver

takluyver Nov 27, 2019

Owner

We should probably throw an error, I'd guess - that means you've given it a name which isn't on PATH, or a path which doesn't exist or isn't executable.

This comment has been minimized.

Copy link
@uranusjr

uranusjr Nov 27, 2019

Author Contributor

I added an exception for this.

@takluyver

This comment has been minimized.

Copy link
Owner

takluyver commented Nov 27, 2019

Thanks, this LGTM.

@takluyver takluyver merged commit 46173cb into takluyver:master Nov 27, 2019
2 of 4 checks passed
2 of 4 checks passed
codecov/patch 40% of diff hit (target 86.36%)
Details
codecov/project 84.63% (-1.73%) compared to 1022269
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants
You can’t perform that action at this time.