Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Long env path names cause weird errors after activiting virtual environment #997

Closed
HaraldNordgren opened this issue Jan 5, 2017 · 14 comments · Fixed by #1579
Closed

Long env path names cause weird errors after activiting virtual environment #997

HaraldNordgren opened this issue Jan 5, 2017 · 14 comments · Fixed by #1579

Comments

@HaraldNordgren
Copy link

@HaraldNordgren HaraldNordgren commented Jan 5, 2017

After creating a virtualenv in the path

/home/harald/git-repos/sandbox/stackoverflow/41454203-how-do-you-import-the-library-being-tested-when-running-a-nosetest/env

I could not run the env's pip,

nosetest/env$ ./bin/pip
bash: ./bin/pip: /home/harald/git-repos/sandbox/stackoverflow/41454203-how-do-you-import-the-l: bad interpreter: Permission denied

and running just pip gives me the system's pip, with all of its packages.

@yan12125

This comment has been minimized.

Copy link
Contributor

@yan12125 yan12125 commented Jan 5, 2017

Not virtualenv's fault. Shebangs have length limits, see http://www.in-ulm.de/~mascheck/various/shebang/.

This is basically a duplicate of #994 - virtualenv are broken with fragile kernel shebang parsing.

@HaraldNordgren

This comment has been minimized.

Copy link
Author

@HaraldNordgren HaraldNordgren commented Jan 5, 2017

Where is the shebang written to file?

And is there maybe some workaround we could employ to make virtual environments more sturdy?

@yan12125

This comment has been minimized.

Copy link
Contributor

@yan12125 yan12125 commented Jan 5, 2017

Where is the shebang written to file?

Do you mean the location of the shebang or the codes that generate a shebang? 1. In the first line of the file bin/pip 2. In distlib

And is there maybe some workaround we could employ to make virtual environments more sturdy?

I don't think there are simple workarounds. A possibility is writing a user space shebang parser just like py.exe on Windows.

@HaraldNordgren

This comment has been minimized.

Copy link
Author

@HaraldNordgren HaraldNordgren commented Jan 5, 2017

Using pip as an example, one thing we could do is move the current pip to .pip, remove the shebang line and create a new pip file with the following content:

#!/bin/bash
/path/to/env/bin/python /path/to/env/bin/.pip $@

Basically, wrap each program in a shell script calling the original file, passing on all of its arguments.

That will get around the problem of long shebangs.

And if we quote these paths in Bash we could probably solve the problem with space characters in paths as well.

@yan12125

This comment has been minimized.

Copy link
Contributor

@yan12125 yan12125 commented Jan 6, 2017

OK maybe my definition for "workarounds" is too strict. I thought it shouldn't break existing things. In this case, using /bin/bash usually works but not always. Some systems do not install bash in /bin/ (FreeBSD install it at /usr/local/bin/bash), and some systems do not provide bash at all. (Most Android uses /system/bin/sh)

A similar idea was proposed 3 years ago: pypa/pip#923 (comment). Apparently no PyPA core developers are interested in this approach :(

@HaraldNordgren

This comment has been minimized.

Copy link
Author

@HaraldNordgren HaraldNordgren commented Jan 6, 2017

No, you're right. The workaround shouldn't break things. As long as env lives in the same place in every Unix system, this should work just as well:

#!/usr/bin/env sh
"/path/to/env/bin/python" "/path/to/env/bin/.pip" $@

It's nice to see that someone else brought up the same idea. However, @cellularmitosis phrased it in negative terms back then, so it only makes sense that no one took it seriously. It doesn't seem like any of the developers actually commented on it.

Let us give it a positive spin this time around and present it as waterproof solution to long paths and spaces. I'm not exactly sure how I would go about to putting this in the virtualenv source code though.

@yan12125

This comment has been minimized.

Copy link
Contributor

@yan12125 yan12125 commented Jan 6, 2017

Looks better. If it's possible to use a single file, I believe it will be ready for a pull request!

@HaraldNordgren

This comment has been minimized.

Copy link
Author

@HaraldNordgren HaraldNordgren commented Jan 9, 2017

@yan12125 I created a pull request.

@vsajip

This comment has been minimized.

Copy link
Contributor

@vsajip vsajip commented May 28, 2017

I've updated distlib to handle long paths and paths with spaces. I didn't use Harald Nordgren's patch directly (it had some problems - e.g. no tests) but the approach is the same - using '/bin/sh' as the executable. pip / virtualenv maintainers might want to test after locally vendoring the current version of the distlib repository.

@cellularmitosis

This comment has been minimized.

Copy link

@cellularmitosis cellularmitosis commented May 28, 2017

@HaraldNordgren ah, I should have been clearer that my dismay was directed at Apple, not at the virtualenv guys! On Linux, this isn't an issue at all, and I had assumed this was an insurmountable shortcoming of OS X shebang parsing. Glad to see there's a solution!

@Imaskar

This comment has been minimized.

Copy link

@Imaskar Imaskar commented Feb 11, 2020

Is it really fixed?
I just run into the same issue using latest pip 20.0.2 and virtualenv 20.0.1 and python 3.7.4 . And it still generates the long path.

@gaborbernat gaborbernat reopened this Feb 11, 2020
@gaborbernat

This comment has been minimized.

Copy link
Contributor

@gaborbernat gaborbernat commented Feb 11, 2020

It seems here we've regressed as we're not using distlib to generate the paths.

@gaborbernat

This comment has been minimized.

Copy link
Contributor

@gaborbernat gaborbernat commented Feb 11, 2020

Hello, a fix for this issue has been released via virtualenv 20.0.2; see https://pypi.org/project/virtualenv/20.0.2/ (https://virtualenv.pypa.io/en/latest/changelog.html#v20-0-2-2020-02-11) . Please give a try and report back if your issue has not been addressed; if not, please comment here, and we'll reopen the ticket. We want to apologize for the inconvenience this has caused you and say thanks for having patience while we resolve the unexpected bugs with this new major release.

thanks

@Imaskar

This comment has been minimized.

Copy link

@Imaskar Imaskar commented Feb 13, 2020

Thank you for acting quick!
I tried 20.0.2 and it worked. It generates full shebang on macOS which supports long paths, and short on Linux.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

6 participants
You can’t perform that action at this time.