unvendor dependencies for 3.0.0

[kennethreitz]

thoughts?

[kennethreitz]

/cc @sigmavirus24 @Lukasa

I'm not sold that it's a good idea. I just like change. Maybe it's time.

[sigmavirus24]

I'm not sold on it being a good idea either. Presently projects may do the following:

# requirements.txt
requests>=2.14.2
urllib3==1.20.0
chardet<2.0.0

In this case, that would break Requests 2.14.2. We'd have to expand our CI to start testing against a wider breadth of our dependencies to allow for users doing something like above. In part, this problem is due to Pip not having a dependency resolver still that would tell the user: Hey, Requests needs at least urllib3 1.21.2 and chardet 2.0.0 so please resolve your dependencies.

At the same time, I'd like us to stop being the project everyone points at and berates because once a year someone on reddit realizes we vendor our dependencies for good reason but is shocked, horrified, mortified, and supposedly going back to httplib.

[Lukasa]

So, I'm open to doing this if and when pip gets a proper dependency solver. A little birdie told me that pip has a GSOC student who is working on doing exactly that, so if that lands I'm open to us considering that change.

[kennethreitz]

hell let's just do it. assigning to myself.

minor version bump.

I'll be pragmatic and require the latest version of urllib3 or greater (or less than the next major).

[kennethreitz]

We'll also emit warnings for incompatible versions at import time.

[sigmavirus24]

We'll also emit warnings for incompatible versions at import time.

Warnings are often ignored by users. This should be a hard exception if we're going to bother with version checks at all.

[kennethreitz]

interesting idea. I like it.

@sigmavirus24 can you provide me with a list of our compatible version ranges, if you're up to helping out with this? Would be much appreciated.

[kennethreitz]

would be nice if Python had an IncompatiblePackages warning. Should we subclass or use RuntimeError?

[sigmavirus24]

The closest thing to an IncompatiblePackages warning is using pkg_resources and by "using" I mean importing it. The problem with that is that importing pkg_resources kicks off a scan of your entire site-packages directory which can be very slow especially if you've done something silly like install several OpenStack services into one site-packages directory (hint, that's not actually that silly).

I don't like subclassing RuntimeError but I can't think of a better option at the moment (then again I'm not really thinking too hard about this).

What I'm worried about is maintaining versions that we use to check for incompatible requirements and maintaining the versions in our setup.py/setup.cfg and how we might do that with the least amount of pain.

[kennethreitz]

I'll find a DRY solution.

[kennethreitz]

Thanks for your thoughts! Much appreciated. I think this will reduce maintinence burden, in an ideal world.

[kennethreitz]

Ian, do you want to get me that list? If not, I'll make another issue for a new contributor to maybe

[kennethreitz]

Meaning, no rush, just wondering :)

[sigmavirus24]

List of dependencies? As they are now,

chardet >= 3.0.2, < 3.1.0
idna >= 2.5, < 2.6
urllib3 >= 1.21.1, < 1.22

That said, I'm struggling to think of a way to cover all of our bases:

1. setup.cfg needs a static listing of requirements (that's not python and thus not easily determined unless we try to generate it as part of our release/build process somehow ... which I don't think we should try)

2. setup.py can use a dynamic list in a file like __version__.py

3. We then need to use the list and dynamically assert things about our dependencies when we import them.

I am thinking what we might want is to make an attempt to go for 90% static package configuration via setup.cfg. It's technically plausible and we could then use something like pkginfo to introspect our selves to find our dependencies. It'll be gross and incur another dependency that will make little sense, but it's a start?

[kennethreitz]

@sigmavirus24 do you want to get a working prototype in place?

[kennethreitz]

related: #4067 4067

[kennethreitz]

Done!