New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pip dependency installation uses unicode __file__ in Python 2 #1525

Closed
pabigot opened this Issue Feb 2, 2014 · 2 comments

Comments

Projects
None yet
3 participants
@pabigot
Copy link

pabigot commented Feb 2, 2014

I recently received a bug report against PyXB that pip would not install pyxb as a dependency in another package. pip has no problems installing pyxb directly. Though blame can be distributed to three places, I'm going to try to convince you pip is the primary place that should be fixed. Here's what's going on:

pip.req.InstallRequirement.run_egg_info builds a script that wraps a package's setup.py. Because pip now understands unicode and the requirement project_name happens to be unicode, it turns out the string stored as setup_py is also unicode, and the generated wrapper includes something like:

__file__ = u'/tmp/pip_build_pab/tester/setup.py'

Now, PyXB's setup.py uses __file__ as a starting point to search for subcomponents that need to be installed. Python os.listdir() will, when given a unicode path, return file names in unicode. Subsequent string manipulations in PyXB preserve this, resulting in unicode package name and paths.

This causes setuptools in Python 2 to reject the installation with:

error in PyXB setup command: package_data must be a dictionary mapping package names to lists of wildcard patterns 

because it has a check_extras function that requires package_data keys and values to be str not basestring instances.

The problem doesn't arise outside of pip, or when pip installs PyXB directly, because then the __file__ that PyXB's setup.py sees is an 8-bit string not a unicode string.

A case could be made that setuptools shouldn't reject unicode package names and paths, but I'm not going to fight that battle. I could also add code to PyXB's setup.py to convert unicode strings, but honestly I don't think it's my problem.

I believe pip shouldn't make __file__ contain a unicode string unless the underlying file system encoding type requires it and you're in Python 3.

pabigot added a commit to pabigot/pip that referenced this issue Feb 6, 2014

fix pypa#1525: unicode __file__ in Python2
The module __file__ attribute in Python2 should be str, not unicode.
When constructing the setup_py value in Python2 use the file system
encoding so repr(setup_py) in scripts does not expand to a unicode
value.

Natim pushed a commit to peopledoc/pip that referenced this issue Feb 19, 2014

Natim pushed a commit to peopledoc/pip that referenced this issue Feb 19, 2014

@Natim

This comment has been minimized.

Copy link

Natim commented Feb 19, 2014

Thx @pabigot
@pypa could you have a look at this PR please ?

Natim pushed a commit to peopledoc/pip that referenced this issue Feb 19, 2014

@dstufft

This comment has been minimized.

Copy link
Member

dstufft commented Feb 20, 2014

This will be fixed with #1538

@dstufft dstufft closed this Feb 20, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment