New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

os.readlink doesn't dereference relative symlinks in mkvirtualenv #268

Open
thom-nic opened this Issue May 11, 2012 · 2 comments

Comments

Projects
None yet
2 participants
@thom-nic
Copy link

thom-nic commented May 11, 2012

So I can't say how or why this problem hasn't occurred for other people, but here's my situation....

I've got python installed via homebrew on Mac. This means (among other things) that /usr/local/bin/python is symlinked to ../Cellar/python/2.7.3/include/python2.7.

Now, when I run virtualenv test, I get a bunch of not-helpful information...

PYTHONHOME is set.  You *must* activate the virtualenv before using it
New python executable in test/bin/python
Traceback (most recent call last):
    File "/usr/local/Cellar/python/2.7.3/lib/python2.7/site.py", line 563, in <module>
        main()
    File "/usr/local/Cellar/python/2.7.3/lib/python2.7/site.py", line 545, in main
        known_paths = addusersitepackages(known_paths)
    File "/usr/local/Cellar/python/2.7.3/lib/python2.7/site.py", line 278, in addusersitepackages
        user_site = getusersitepackages()
    File "/usr/local/Cellar/python/2.7.3/lib/python2.7/site.py", line 253, in getusersitepackages
        user_base = getuserbase() # this will also set USER_BASE
    File "/usr/local/Cellar/python/2.7.3/lib/python2.7/site.py", line 243, in getuserbase
        USER_BASE = get_config_var('userbase')
    File "/usr/local/Cellar/python/2.7.3/lib/python2.7/sysconfig.py", line 521, in get_config_var
        return get_config_vars().get(name)
    File "/usr/local/Cellar/python/2.7.3/lib/python2.7/sysconfig.py", line 420, in get_config_vars
        _init_posix(_CONFIG_VARS)
    File "/usr/local/Cellar/python/2.7.3/lib/python2.7/sysconfig.py", line 299, in _init_posix
        raise IOError(msg)
IOError: invalid Python installation: unable to open /Users/tnichols/.virtualenvs/test/include/python2.7/pyconfig.h (No such file or directory)
ERROR: The executable test/bin/python is not functioning
ERROR: It thinks sys.prefix is u'/Users/tnichols/.virtualenvs' (should be u'/Users/tnichols/.virtualenvs/test')
ERROR: virtualenv is not compatible with this system or executable

But believe it or not, the problem is that the copied files are broken symlinks:

$ ll ~/.virtualenvs/test/include/python2.7
lrwxr-x---  1 tnichols  2088035466    40B May 11 10:28 /Users/tnichols/.virtualenvs/test/include/python2.7@ -> ../Cellar/python/2.7.3/include/python2.7

Digging in to virtualenv.py, I found line 423 of the copyfile function, where it appears you're trying to dereference symlinks by doing the following:

if not os.path.islink(src):
    srcpath = os.path.abspath(src)
else:
    srcpath = os.readlink(src)

However this isn't achieving the desired function, at least not in my environment...

$ python
Python 2.7.3 (default, May 11 2012, 09:48:41) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.path.realpath('/usr/local/bin/python')
'/usr/local/Cellar/python/2.7.3/bin/python2.7'
>>> os.readlink('/usr/local/bin/python')
'../Cellar/python/2.7.3/bin/python'
>>> os.path.abspath('/usr/local/bin/python')
'/usr/local/bin/python'

So readlink is returning the relative path that the link is using. If I change the line to srcpath = os.path.realpath(src) then it dereferences the link and converts it to an absolute path that resolves correctly!

@atsampson

This comment has been minimized.

Copy link
Contributor

atsampson commented Oct 7, 2012

I've just run into this bug as well (while building Firefox on a system where Python's installed using stow) -- copyfile is broken for symlinks that point to relative paths, and for symlinks that point to further symlinks. This patch makes it follow symlinks until it finds a file:

--- work/mozilla-release/other-licenses/virtualenv/virtualenv.py.orig   2012-10-07 22:34:22.964000001 +0100
+++ work/mozilla-release/other-licenses/virtualenv/virtualenv.py        2012-10-07 22:37:38.532000000 +0100
@@ -417,10 +417,10 @@
     if not os.path.exists(os.path.dirname(dest)):
         logger.info('Creating parent directories for %s' % os.path.dirname(dest))
         os.makedirs(os.path.dirname(dest))
-    if not os.path.islink(src):
-        srcpath = os.path.abspath(src)
-    else:
-        srcpath = os.readlink(src)
+    srcpath = src
+    while os.path.islink(srcpath):
+        srcpath = os.path.join(os.path.dirname(srcpath), os.readlink(srcpath))
+    srcpath = os.path.abspath(srcpath)
     if symlink and hasattr(os, 'symlink') and not is_win:
         logger.info('Symlinking %s', dest)
         try:

atsampson added a commit to atsampson/virtualenv that referenced this issue Sep 20, 2016

Dereference symlinks more thoroughly in copyfile
copyfile was broken for symlinks that pointed to relative paths, and for
symlinks that pointed to further symlinks (which meant that virtualenv
failed to work in some cases when the Python executable it found was
a symlink, e.g. when Python had been installed using stow or homebrew;
issue pypa#268).

Make it follow symlinks until it finds an actual file to copy.

atsampson added a commit to atsampson/virtualenv that referenced this issue Dec 17, 2018

Dereference symlinks more thoroughly in copyfile
copyfile was broken for symlinks that pointed to relative paths, and for
symlinks that pointed to further symlinks (which meant that virtualenv
failed to work in some cases when the Python executable it found was
a symlink, e.g. when Python had been installed using stow or homebrew;
issue pypa#268).

Make it follow symlinks until it finds an actual file to copy.

atsampson added a commit to atsampson/virtualenv that referenced this issue Dec 17, 2018

Dereference symlinks more thoroughly in copyfile
copyfile was broken for symlinks that pointed to relative paths, and for
symlinks that pointed to further symlinks (which meant that virtualenv
failed to work in some cases when the Python executable it found was
a symlink, e.g. when Python had been installed using stow or homebrew;
issue pypa#268).

Make it follow symlinks until it finds an actual file to copy.

atsampson added a commit to atsampson/virtualenv that referenced this issue Dec 17, 2018

Dereference symlinks more thoroughly in copyfile
copyfile was broken for symlinks that pointed to relative paths, and for
symlinks that pointed to further symlinks (which meant that virtualenv
failed to work in some cases when the Python executable it found was
a symlink, e.g. when Python had been installed using stow or homebrew;
issue pypa#268).

Make it follow symlinks until it finds an actual file to copy.

atsampson added a commit to atsampson/virtualenv that referenced this issue Dec 17, 2018

Dereference symlinks more thoroughly in copyfile
copyfile was broken for symlinks that pointed to relative paths, and for
symlinks that pointed to further symlinks (which meant that virtualenv
failed to work in some cases when the Python executable it found was
a symlink, e.g. when Python had been installed using stow or homebrew;
issue pypa#268).

Make it follow symlinks until it finds an actual file to copy.

atsampson added a commit to atsampson/virtualenv that referenced this issue Dec 17, 2018

Dereference symlinks more thoroughly in copyfile
copyfile was broken for symlinks that pointed to relative paths, and for
symlinks that pointed to further symlinks (which meant that virtualenv
failed to work in some cases when the Python executable it found was
a symlink, e.g. when Python had been installed using stow or homebrew;
issue pypa#268).

Make it follow symlinks until it finds an actual file to copy.

atsampson added a commit to atsampson/virtualenv that referenced this issue Dec 17, 2018

Dereference symlinks more thoroughly in copyfile
copyfile was broken for symlinks that pointed to relative paths, and for
symlinks that pointed to further symlinks (which meant that virtualenv
failed to work in some cases when the Python executable it found was
a symlink, e.g. when Python had been installed using stow or homebrew;
issue pypa#268).

Make it follow symlinks until it finds an actual file to copy.

atsampson added a commit to atsampson/virtualenv that referenced this issue Dec 17, 2018

Dereference symlinks more thoroughly in copyfile
copyfile was broken for symlinks that pointed to relative paths, and for
symlinks that pointed to further symlinks (which meant that virtualenv
failed to work in some cases when the Python executable it found was
a symlink, e.g. when Python had been installed using stow or homebrew;
issue pypa#268).

Make it follow symlinks until it finds an actual file to copy.

atsampson added a commit to atsampson/virtualenv that referenced this issue Dec 17, 2018

Dereference symlinks more thoroughly in copyfile
copyfile was broken for symlinks that pointed to relative paths, and for
symlinks that pointed to further symlinks (which meant that virtualenv
failed to work in some cases when the Python executable it found was
a symlink, e.g. when Python had been installed using stow or homebrew;
issue pypa#268).

Make it follow symlinks until it finds an actual file to copy.

atsampson added a commit to atsampson/virtualenv that referenced this issue Dec 17, 2018

Dereference symlinks more thoroughly in copyfile
copyfile was broken for symlinks that pointed to relative paths, and for
symlinks that pointed to further symlinks (which meant that virtualenv
failed to work in some cases when the Python executable it found was
a symlink, e.g. when Python had been installed using stow or homebrew;
issue pypa#268).

Make it follow symlinks until it finds an actual file to copy.

atsampson added a commit to atsampson/virtualenv that referenced this issue Dec 17, 2018

Dereference symlinks more thoroughly in copyfile
copyfile was broken for symlinks that pointed to relative paths, and for
symlinks that pointed to further symlinks (which meant that virtualenv
failed to work in some cases when the Python executable it found was
a symlink, e.g. when Python had been installed using stow or homebrew;
issue pypa#268).

Make it follow symlinks until it finds an actual file to copy.

gaborbernat added a commit that referenced this issue Dec 18, 2018

Dereference symlinks more thoroughly in copyfile (#956)
copyfile was broken for symlinks that pointed to relative paths, and for
symlinks that pointed to further symlinks (which meant that virtualenv
failed to work in some cases when the Python executable it found was
a symlink, e.g. when Python had been installed using stow or homebrew;
issue #268).

Make it follow symlinks until it finds an actual file to copy.
@stale

This comment has been minimized.

Copy link

stale bot commented Jan 15, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Just add a comment if you want to keep it open. Thank you for your contributions.

@stale stale bot added the wontfix label Jan 15, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment