New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pkg_resources.find_on_path() thinks the zc.recipe.egg wheel is an egg #462

Closed
bb-migration opened this Issue Nov 18, 2015 · 14 comments

Comments

Projects
None yet
4 participants
@bb-migration

bb-migration commented Nov 18, 2015

Originally reported by: mgedmin (Bitbucket: mgedmin, GitHub: mgedmin)


Steps to reproduce: try to pip install zc.recipe.egg. You'll get an error:

zc.recipe.egg is in an unsupported or invalid wheel

If you hack pip's sources to expose the real traceback, you'll see this:

Traceback (most recent call last):                                                                                                                                                                                                           
  File "/home/qwcode/p/pypa/pip/pip/basecommand.py", line 200, in main                                                                                                                                                                       
    status = self.run(options, args)                                                                                                                                                                                                         
  File "/home/qwcode/p/pypa/pip/pip/commands/install.py", line 302, in run                                                                                                                                                                   
    root=options.root_path,                                                                                                                                                                                                                  
  File "/home/qwcode/p/pypa/pip/pip/req/req_set.py", line 657, in install                                                                                                                                                                    
    **kwargs
  File "/home/qwcode/p/pypa/pip/pip/req/req_install.py", line 805, in install
    version = pip.wheel.wheel_version(self.source_dir)
  File "/home/qwcode/p/pypa/pip/pip/wheel.py", line 559, in wheel_version
    wheel_data = dist.get_metadata('WHEEL')
  File "/home/qwcode/p/pypa/pip/pip/_vendor/pkg_resources/__init__.py", line 1611, in get_metadata
    return self._get(self._fn(self.egg_info, name))
  File "/home/qwcode/p/pypa/pip/pip/_vendor/pkg_resources/__init__.py", line 1722, in _get
    with open(path, 'rb') as stream:
IOError: [Errno 2] No such file or directory: '/tmp/pip-build-ucnzjI/zc.recipe.egg/EGG-INFO/WHEEL'

AFAIU the problem is the following check in pkg_resources.find_on_path():

    if os.path.isdir(path_item) and os.access(path_item, os.R_OK):
        if path_item.lower().endswith('.egg'):
            # unpacked egg

This is wrong: the unpacked zc.recipe.egg wheel has a top-level directory called zc.recipe.egg.

See pypa/pip#3028 for more background.


@bb-migration

This comment has been minimized.

bb-migration commented Nov 18, 2015

Original comment by mgedmin (Bitbucket: mgedmin, GitHub: mgedmin):


Proposed fix: change the inner check to

        if path_item.lower().endswith('.egg') and os.path.isdir(os.path.join(path_item, 'EGG-INFO')):
@bb-migration

This comment has been minimized.

bb-migration commented Nov 18, 2015

Original comment by iElectric (Bitbucket: iElectric, GitHub: Unknown):


I've tried that and zc.recipe.egg will still claim to be unsupported wheel.

@bb-migration

This comment has been minimized.

bb-migration commented Nov 18, 2015

Original comment by iElectric (Bitbucket: iElectric, GitHub: Unknown):


Note that it occurs at multiple places:

    substituteInPlace pkg_resources/__init__.py \
      --replace "if path_item.lower().endswith('.egg'):" "if path_item.lower().endswith('.egg') and os.path.isdir(os.path.join(path_item, 'EGG-INFO')):" \
      --replace "if subitem.endswith('.egg'):" "if subitem.endswith('.egg') and os.path.isdir(os.path.join(subitem, 'EGG-INFO')):" \
      --replace "if path.lower().endswith('.egg'):" "if path.lower().endswith('.egg') and os.path.isdir(os.path.join(path, 'EGG-INFO')):" \
      --replace "lower.endswith('.egg')" "lower.endswith('.egg') and os.path.isdir(os.path.join(lower, 'EGG-INFO'))"
@bb-migration

This comment has been minimized.

bb-migration commented Nov 23, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


If making these changes addresses the issue, I have a patch staged and will try to push tomorrow.

@bb-migration

This comment has been minimized.

bb-migration commented Nov 23, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


Only detect a path as an unpacked egg if it also has an EGG-INFO directory. Fixes #462.

@bb-migration

This comment has been minimized.

bb-migration commented Nov 23, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


Extract function for detecting unpacked egg. Ref #462.

@bb-migration

This comment has been minimized.

bb-migration commented Nov 24, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


Back out 435b1b762fba. The change breaks detection of zip eggs. Reopens #462.

@bb-migration

This comment has been minimized.

bb-migration commented Nov 24, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


It seems that some of those tests for endswith('.egg') are also expected to match when the target is a zipped egg, and when that detection fails, it causes problems when installing zipped eggs (setuptools tests themselves fail to run).

@bb-migration

This comment has been minimized.

bb-migration commented Dec 4, 2015

Original comment by datakurre (Bitbucket: datakurre, GitHub: datakurre):


Are zipped eggs just files? If so, maybe

#!python

and (os.path.isfile(path)  # is zipped egg
     or os.path.isdir(os.path.join(path, 'EGG-INFO'))

@mgedmin mgedmin referenced this issue Feb 20, 2017

Merged

Humpty had a great fall #9

0 of 1 task complete
@tseaver

This comment has been minimized.

Contributor

tseaver commented Feb 20, 2017

The stable build of the Zope RTD docs just now failed due to this bug.

@pradyunsg

This comment has been minimized.

Member

pradyunsg commented Jun 14, 2017

@jaraco Would updating just a bunch of conditionals be enough to fix this?

leorochael added a commit to leorochael/setuptools that referenced this issue Jul 31, 2017

Better detect unpacked eggs
Do not assume a directory named in `.egg` is an egg, unless it has an
actual egg metadata directory.

Closes pypa#462
@leorochael

This comment has been minimized.

Contributor

leorochael commented Jul 31, 2017

Would appreciate reviewers for #1116, which I'm proposing to fix this issue.

leorochael added a commit to leorochael/setuptools that referenced this issue Jul 31, 2017

Better detect unpacked eggs
Do not assume a directory named in `.egg` is an egg, unless it has an
actual egg metadata directory.

Closes pypa#462

leorochael added a commit to leorochael/setuptools that referenced this issue Jul 31, 2017

Better detect unpacked eggs
Do not assume a directory named in `.egg` is an egg, unless it has an
actual egg metadata directory.

Closes pypa#462

@jaraco jaraco closed this in #1116 Jul 31, 2017

@leorochael

This comment has been minimized.

Contributor

leorochael commented Aug 1, 2017

I think this issue was auto-closed due to #1116 merging.

I suggest keeping it open until pip revendors the latest setuptools.

@leorochael

This comment has been minimized.

Contributor

leorochael commented Aug 1, 2017

Sorry, nevermind. Confused this with the related pip issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment