Skip to content
This repository

Creation of a new virtuelenv with Python 3.3 fails #329

Closed
michaelklishin opened this Issue · 31 comments

6 participants

Michael Klishin Jannis Leidel Paul Nasrat Tomi Pieviläinen Carl Meyer Gabriel de Perthuis
Michael Klishin

When trying to provision 3.3 RC1 [upd.: and later RC2] for travis-ci.org, I am hitting the following (Chef output):
https://gist.github.com/c47e2107af831c83391b

I am not familiar with the Python ecosystem enough to understand if this qualifies as a virtualenv problem but decided to file this issue. If there are workarounds, I think we can tweak things just for 3.3.

Jannis Leidel
Owner

Is that the latest version of virtualenv/pip?

Paul Nasrat
Collaborator

This looks similar to the issues I addressed on OS X for 3.3, which should be fixed in latest virtualenv/pip.

You probably need the virtualenv on the base image to be 1.8.1 - if you need a PPA or whatever for that, then I'm not sure where to look but it should be easy to package.

ISTR It's possible to create a travis vagrant instance from your OSS repos - can you point me at a scriptable reproducer and I can poke more.

Paul Nasrat
Collaborator

I'm going to hop on IRC to see if we can get a quicker turnaround

Paul Nasrat
Collaborator

Full logs: https://gist.github.com/3dd90923365081565e9e

Looks like possibly something odd in the new ubuntu packaging with the minimal package

PKG source https://launchpad.net/~python-dev/+archive/ppa

  Traceback (most recent call last):
    File "setup.py", line 220, in <module>
      scripts = scripts,
    File "/usr/lib/python3.3/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.3/distutils/dist.py", line 917, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.3/distutils/dist.py", line 935, in run_command
      cmd_obj.ensure_finalized()
    File "/usr/lib/python3.3/distutils/cmd.py", line 107, in ensure_finalized
      self.finalize_options()
    File "build/src/setuptools/command/install.py", line 29, in finalize_options
      _install.finalize_options(self)
    File "/usr/lib/python3.3/distutils/command/install.py", line 334, in finalize_options
      (prefix, exec_prefix) = get_config_vars('prefix', 'exec_prefix')
    File "/home/travis/virtualenv/python3.3/lib/python3.3/distutils/__init__.py", line 92, in sysconfig_get_config_vars
      real_vars = old_get_config_vars(*args)
    File "/usr/lib/python3.3/distutils/sysconfig.py", line 505, in get_config_vars
      func()
    File "/usr/lib/python3.3/distutils/sysconfig.py", line 449, in _init_posix
      from _sysconfigdata import build_time_vars
    File "/usr/lib/python3.3/_sysconfigdata.py", line 6, in <module>
      from _sysconfigdata_m import *
  ImportError: No module named '_sysconfigdata_m'
Michael Klishin

The package we use wasn't very heavily tested but the maintainer is very kind and I think we can ask him to rebuild it if we identify what is missing/wrong.

Tomi Pieviläinen

You are hitting virtualenv bug #301 in combination of either http://bugs.python.org/issue15822 (3.3rc1) or http://bugs.python.org/issue15890 (rc2).

Carl Meyer
Owner

Thanks @tpievila for the additional pointers. #301 is an error-reporting/debugging improvement, not something that will cause a failure itself; so if that's correct this is not something we can fix in virtualenv. Fortunately it looks like http://bugs.python.org/issue15822 is fixed in 3.3rc2, and http://bugs.python.org/issue15890 is not hard to work around if needed. Closing for now, unless more information is provided indicating this is a virtualenv bug.

Michael Klishin

@carljm does it mean that using RC2 will solve the issue? We are not Python experts and working around Python issues is not necessarily something we can do. We would also prefer to not modify virtualenv in any way.

Michael Klishin

The issue is still present with Python 3.3 RC2. Log with RC2. What our next steps should be? We cannot provide Python 3.3 on travis because of this :( Maybe some Python hackers would be interested in helping us a little bit?

Michael Klishin

We use a different PPA for 3.3 this time, by the way.

Carl Meyer
Owner

http://bugs.python.org/issue15822 should be fixed in rc2, and on further review it seems unlikely to me that you are hitting http://bugs.python.org/issue15890, since the consensus on that bug seems to be that it's not a regression, so if your umask was set too strictly you'd have seen that problem on earlier Pythons as well (also, not likely that a PPA build would be broken in that way). So it seems likely that @tpievila 's analysis was incorrect.

Reviewing this thread, I don't see confirmation anywhere of the version of virtualenv/pip you are using, @michaelklishin ?

Carl Meyer carljm reopened this
Michael Klishin

pip 1.2.1, virtualenv 1.8.2.

Carl Meyer
Owner

@michaelklishin I notice in your log that you're running virtualenv over top of an existing virtualenv directory; can you try rm -rf /home/travis/virtualenv/python3.3 and then rerun the virtualenv-creation? Also, can you run with -vvv? Thanks.

(If you're working on this now, I'm also carljm on Freenode in the #pip IRC channel.)

Paul Nasrat
Collaborator

http://packages.ubuntu.com/quantal/i386/libpython3.3-minimal/filelist

Gives /usr/lib/python3.3/plat-i386-linux-gnu/_sysconfigdata_m.py I'm betting that's not in sys.path

https://wiki.ubuntu.com/MultiarchSpec

Michael Klishin
travis@precise32:~$ ls -lh /usr/lib/python3.3/plat-i386-linux-gnu/
total 32K
-rw-r--r-- 1 root root 5.9K Sep 17 14:57 IN.py
drwxr-xr-x 2 root root 4.0K Sep 17 17:31 __pycache__
-rw-r--r-- 1 root root  19K Sep 17 14:58 _sysconfigdata_m.py
Paul Nasrat
Collaborator

Note for self/others - http://bazaar.launchpad.net/~doko/python/pkg3.3-debian/files has the patches, etc.

Michael Klishin

@mezzle suggests there is a patch in Quantal that our package did not include. We are waiting for another build to finish to try it out with supposedly full patch set.

Paul Nasrat pnasrat reopened this
Paul Nasrat
Collaborator

oops fat fingered.

Carl Meyer
Owner

@michaelklishin What is the value of sys.path inside the virtualenv? (i.e. /home/travis/virtualenv/python3.3/bin/python -c "import sys; print(sys.path)")

Michael Klishin
travis@precise32:~$ /home/travis/virtualenv/python3.3/bin/python -c "import sys; print(sys.path)"
['', '/home/travis/virtualenv/python3.3/lib/python3.3', '/home/travis/virtualenv/python3.3/lib/python3.3/plat-i386-linux-gnu', '/home/travis/virtualenv/python3.3/lib/python3.3/lib-dynload', '/usr/lib/python3.3', '/home/travis/virtualenv/python3.3/lib/python3.3/site-packages']
Carl Meyer
Owner

Yes, @pnasrat 's hunch was correct; we need to add a new special case in virtualenv's site.py to check if we're on a system using this multi-arch spec and add the system multi-arch-specific plat dir to sys.path. Thanks @michaelklishin for the debugging help.

Paul Nasrat
Collaborator

I'm guessing this is still occuring - @carljm do you have time for this, else I'lll try take a poke on the weekend

Carl Meyer
Owner

@pnasrat If I have time it won't be until mid-next-week at the earliest; if you have time this weekend that'd be great.

Paul Nasrat
Collaborator

Sure - I should have time then, I'm at conference during the week.

Michael Klishin

Any progress on this? can someone who is not a virtualenv core contributor submit a fix? travis-ci.org users are already asking for 3.3.

Gabriel de Perthuis
g2p commented

@carljm I think it's more a question of symlinking the plat-$(MULTIARCH) dir when building the virtualenv.
The path is built correctly where I am (from Quantal's python3.3, which travis will get backported in due course), but the directory it refers to doesn't exist. It's part of the platform-dependent sys.path default, which exists before site.py files are read (confirmed with python3.3 -ESsc 'import sys; print(sys.path)' inside and outside the virtualenv).

Michael Klishin

Were there any Virtualenv releases that include this fix? We still cannot provide Python 3.3 on travis-ci.org because of this issue.

Carl Meyer
Owner

I don't think any releases have included this fix yet. It could certainly be worth a bugfix release (@jezdez?) but in the meantime you should be able to use master virtualenv without negative consequences.

Paul Nasrat
Collaborator

+! for a release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.