Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pkg_resources.find_on_path() thinks the zc.recipe.egg wheel is an egg #462

Closed
ghost opened this issue Nov 18, 2015 · 14 comments · Fixed by #1116
Closed

pkg_resources.find_on_path() thinks the zc.recipe.egg wheel is an egg #462

ghost opened this issue Nov 18, 2015 · 14 comments · Fixed by #1116

Comments

@ghost
Copy link

ghost commented Nov 18, 2015

Originally reported by: mgedmin (Bitbucket: mgedmin, GitHub: mgedmin)


Steps to reproduce: try to pip install zc.recipe.egg. You'll get an error:

zc.recipe.egg is in an unsupported or invalid wheel

If you hack pip's sources to expose the real traceback, you'll see this:

Traceback (most recent call last):                                                                                                                                                                                                           
  File "/home/qwcode/p/pypa/pip/pip/basecommand.py", line 200, in main                                                                                                                                                                       
    status = self.run(options, args)                                                                                                                                                                                                         
  File "/home/qwcode/p/pypa/pip/pip/commands/install.py", line 302, in run                                                                                                                                                                   
    root=options.root_path,                                                                                                                                                                                                                  
  File "/home/qwcode/p/pypa/pip/pip/req/req_set.py", line 657, in install                                                                                                                                                                    
    **kwargs
  File "/home/qwcode/p/pypa/pip/pip/req/req_install.py", line 805, in install
    version = pip.wheel.wheel_version(self.source_dir)
  File "/home/qwcode/p/pypa/pip/pip/wheel.py", line 559, in wheel_version
    wheel_data = dist.get_metadata('WHEEL')
  File "/home/qwcode/p/pypa/pip/pip/_vendor/pkg_resources/__init__.py", line 1611, in get_metadata
    return self._get(self._fn(self.egg_info, name))
  File "/home/qwcode/p/pypa/pip/pip/_vendor/pkg_resources/__init__.py", line 1722, in _get
    with open(path, 'rb') as stream:
IOError: [Errno 2] No such file or directory: '/tmp/pip-build-ucnzjI/zc.recipe.egg/EGG-INFO/WHEEL'

AFAIU the problem is the following check in pkg_resources.find_on_path():

    if os.path.isdir(path_item) and os.access(path_item, os.R_OK):
        if path_item.lower().endswith('.egg'):
            # unpacked egg

This is wrong: the unpacked zc.recipe.egg wheel has a top-level directory called zc.recipe.egg.

See pypa/pip#3028 for more background.


@ghost
Copy link
Author

ghost commented Nov 18, 2015

Original comment by mgedmin (Bitbucket: mgedmin, GitHub: mgedmin):


Proposed fix: change the inner check to

        if path_item.lower().endswith('.egg') and os.path.isdir(os.path.join(path_item, 'EGG-INFO')):

@ghost
Copy link
Author

ghost commented Nov 18, 2015

Original comment by iElectric (Bitbucket: iElectric, GitHub: Unknown):


I've tried that and zc.recipe.egg will still claim to be unsupported wheel.

@ghost
Copy link
Author

ghost commented Nov 18, 2015

Original comment by iElectric (Bitbucket: iElectric, GitHub: Unknown):


Note that it occurs at multiple places:

    substituteInPlace pkg_resources/__init__.py \
      --replace "if path_item.lower().endswith('.egg'):" "if path_item.lower().endswith('.egg') and os.path.isdir(os.path.join(path_item, 'EGG-INFO')):" \
      --replace "if subitem.endswith('.egg'):" "if subitem.endswith('.egg') and os.path.isdir(os.path.join(subitem, 'EGG-INFO')):" \
      --replace "if path.lower().endswith('.egg'):" "if path.lower().endswith('.egg') and os.path.isdir(os.path.join(path, 'EGG-INFO')):" \
      --replace "lower.endswith('.egg')" "lower.endswith('.egg') and os.path.isdir(os.path.join(lower, 'EGG-INFO'))"

@ghost
Copy link
Author

ghost commented Nov 23, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


If making these changes addresses the issue, I have a patch staged and will try to push tomorrow.

@ghost
Copy link
Author

ghost commented Nov 23, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


Only detect a path as an unpacked egg if it also has an EGG-INFO directory. Fixes #462.

@ghost
Copy link
Author

ghost commented Nov 23, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


Extract function for detecting unpacked egg. Ref #462.

@ghost
Copy link
Author

ghost commented Nov 24, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


Back out 435b1b762fba. The change breaks detection of zip eggs. Reopens #462.

@ghost
Copy link
Author

ghost commented Nov 24, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


It seems that some of those tests for endswith('.egg') are also expected to match when the target is a zipped egg, and when that detection fails, it causes problems when installing zipped eggs (setuptools tests themselves fail to run).

@ghost
Copy link
Author

ghost commented Dec 4, 2015

Original comment by datakurre (Bitbucket: datakurre, GitHub: datakurre):


Are zipped eggs just files? If so, maybe

#!python

and (os.path.isfile(path)  # is zipped egg
     or os.path.isdir(os.path.join(path, 'EGG-INFO'))

@tseaver
Copy link
Contributor

tseaver commented Feb 20, 2017

The stable build of the Zope RTD docs just now failed due to this bug.

@pradyunsg
Copy link
Member

@jaraco Would updating just a bunch of conditionals be enough to fix this?

leorochael added a commit to leorochael/setuptools that referenced this issue Jul 31, 2017
Do not assume a directory named in `.egg` is an egg, unless it has an
actual egg metadata directory.

Closes pypa#462
@leorochael
Copy link
Contributor

Would appreciate reviewers for #1116, which I'm proposing to fix this issue.

leorochael added a commit to leorochael/setuptools that referenced this issue Jul 31, 2017
Do not assume a directory named in `.egg` is an egg, unless it has an
actual egg metadata directory.

Closes pypa#462
leorochael added a commit to leorochael/setuptools that referenced this issue Jul 31, 2017
Do not assume a directory named in `.egg` is an egg, unless it has an
actual egg metadata directory.

Closes pypa#462
@leorochael
Copy link
Contributor

I think this issue was auto-closed due to #1116 merging.

I suggest keeping it open until pip revendors the latest setuptools.

@leorochael
Copy link
Contributor

Sorry, nevermind. Confused this with the related pip issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants