warn user when requested extra does not exist - 2138

[derwolfe]

Working on adding loggers for the pkg_resources.UnknonwExtra exception that can be thrown when a user attempts to install an unknown extra.

[derwolfe]

This is in reference to issue #2138

[dstufft]

I'm actually confused a little bit about how this is working right now. It appears that pip install requests[nonexistent] fails right now because it installs from a Wheel, but pip install --no-use-wheel requests[nonexistant] doesn't fail from an sdist (and doesn't print any kind of message either).

Since this code looks like it takes a failure and turns it into a warning I'm guessing that this is only going to take affect for Wheel installs. It would be great to unify this so that we raise the same warning in both cases.

[derwolfe]

Sorry about that dstufft, I meant to put a label on the PR that this was in progress. In effect, this PR doesn't do anything right now.

I mainly made the PR so that I could use travis.

[msabramo]

@derwolfe: Are you planning to update this?

[derwolfe]

@msabramo - yes, I am planning on working on it this week before Friday.

[derwolfe]

This is related to PR #2153. Once it lands, the job of this PR will (should) be very simple to implement. Most of the legwork is really done by the pkg_resources.Distribution.requires() function, which that PR enables for non-wheel installs.

[derwolfe]

Sine PR #2153 has landed, tbis should be pretty simple to implement. I'm on vacation now, but I will be able to start working on this again on Dec 28.

[derwolfe]

Still working on this, albeit very slowly. I've rebased against develop so the #2153 changes are included.

Currently, if a non existent option is passed in and a single exception is thrown, This means if a user has a list of options like

pip install twisted[zorro, tls]

where tls actually exists; because of zorro failing, the tls option will not be installed.

In my mind the correct behaviour should be: try to install every option, if the option doesn't exist, log it, and move on to the next. @dstufft: does this proposed behaviour seem reasonable to you?

[derwolfe]

Ready for review :-).

[xavfernandez]

The static methods seem a bit much for basically doing

set(req_to_install.extras) & set(dist.extras)

and

    set(req_to_install.extras) - set(dist.extras)

[derwolfe]

I agree, I hadn't though about using sets. I will update the code to use this and will remove the static methods and unit tests covering them. Thanks for your feedback.

[derwolfe]

@xavfernandez - Though, there is an issue with sets not guaranteeing the order of their items. Would a decent compromise be to just inline the comprehensions? This would still remove the static methods and their unit tests.

[msabramo]

You could always sort the set items if you need guaranteed ordering for tests or something.

[derwolfe]

True - I could just do sorted(missing). Thanks!

[derwolfe]

I've integrated @xavfernandez's suggested changes into the patch.
It looks like travis only failed due to a connection error installing dependencies.

If anyone can restart the build, it is ready for re-review.

[sigmavirus24]

Mind if I ask for a bit of style consistency? You have

lval = func(
   args
)

And

obj.method(
    args,
    more_args)

[msabramo]

nonexistant => nonexistent

[derwolfe]

@msabramo - thanks for your comments. I've made the changes you requested.

[rbu]

Also fixes #852 ?

[msabramo]

Nice.

$ pip install 'bpython[nonexistent]'
Collecting bpython[nonexistent]
  Using cached bpython-0.14.1-py2.py3-none-any.whl
  bpython 0.14.1 does not provide the extra 'nonexistent'
...

$ pip install --no-use-wheel 'bpython[nonexistent]'
Requirement already satisfied (use --upgrade to upgrade): bpython[nonexistent] in /Users/marca/python/virtualenvs/pip/lib/python2.7/site-packages
  bpython 0.14.1 does not provide the extra 'nonexistent'

It's nice that it's a warning instead of a fatal error with a traceback.

I did run into a problem where it couldn't figure out the name of the package when I tried it with sentry (the example in #852):

$ pip install -U 'sentry[lol]'
...
  Unknown 7.4.1 does not provide the extra 'lol'

Note how it say "Unknown" instead of "sentry".

Although to be fair, the current code on develop has the same problem:

$ pip install -U 'sentry[lol]'
Collecting sentry[lol]
  Using cached sentry-7.4.1.tar.gz
  Exception:
  Traceback (most recent call last):
    File "/Users/marca/dev/git-repos/pip/pip/basecommand.py", line 232, in main
      status = self.run(options, args)
    File "/Users/marca/dev/git-repos/pip/pip/commands/install.py", line 342, in run
      requirement_set.prepare_files(finder)
    File "/Users/marca/dev/git-repos/pip/pip/req/req_set.py", line 427, in prepare_files
      req_to_install.extras):
    File "/Users/marca/dev/git-repos/pip/pip/_vendor/pkg_resources/__init__.py", line 2504, in requires
      "%s has no such extra feature %r" % (self, ext)
  UnknownExtra: Unknown 7.4.1 has no such extra feature 'lol'

[msabramo]

$ pip install -U 'sentry[lol]'
Collecting sentry[lol]
  Using cached sentry-7.4.1.tar.gz
processing ~/.pdbrc.py
[5] > /Users/marca/dev/git-repos/pip/pip/_vendor/pkg_resources/__init__.py(2504)requires()
-> raise UnknownExtra(
(Pdb++) list 2504, 3
2504 ->                  raise UnknownExtra(
2505                       "%s has no such extra feature %r" % (self, ext)
2506                   )
2507           return deps
(Pdb++) self
Unknown 7.4.1 (/private/var/folders/gw/w0clrs515zx9x_55zgtpv4mm0000gp/T/pip-build-zo2eu1/sentry/pip-egg-info/sentry.egg-info)

[msabramo]

OK, I think I found a fix for the above issue. I'll file a separate PR for it.

[derwolfe]

@msabramo - Great! When that error is fixed, some of my new tests can be updated as well.

[msabramo]

Here's the PR I mentioned: #2530

[msabramo]

If I merge the change here with the change in #2530:

$ git log -n 1
commit 631a1827806822875862db520deaf2460cd60c5d
Merge: 97b03f4 a12ecdf
Author: Marc Abramowitz <marc@marc-abramowitz.com>
Date:   Thu Mar 12 12:57:57 2015 -0700

    Merge branch 'get_dist_handle_when_egg_info_path_has_trailing_slash' into pr_2142_get_dist

then I get correct behavior:

$ pip install -U 'sentry[lol]'
Collecting sentry[lol]
  Using cached sentry-7.4.1.tar.gz
  sentry 7.4.1 does not provide the extra 'lol'
...

[msabramo]

Also fixes #852 ?

Yes. I believe it does.