Consider dropping/deprecating Python 2.6 support

[bitprophet]

Background

There are a number of arguments among progressive package maintainers for dropping 2.6:

• Not supported by the Python core team (including security bugs)
• Increasing numbers of third-party packages are also dropping support or at least doing this same sort of side-eye at the possibility.
  • Including pip, which isn't really "third party", and tends to drag the rest of us screaming along with it
  • Sphinx, as of late 2016, has also dropped Python 2.6 support
• Debian 7 and up include 2.7; RHEL/CentOS 7 too
  • Users still stuck on RHEL5 must be backporting their own Python if they're on 2.6 (as system default was 2.4) so they arguably can just as easily get 2.7
• Usual "backporting packages is a core need for good systems administration, as is not using unsupported software" argument
• (Admittedly minor, IMHO) syntax niceties like "you can use empty brace expressions in strings, dict comprehensions, etc"

But there are strong counter-arguments, which is why we currently still support 2.6:

• Through recently, a nontrivial number of users (across my personal maintained packages) are still on 2.6 - as high as 25-33% for Paramiko for example.
  • EDIT: though we just re-ran the numbers and it's now at only 6%! 60k downloads over 2 weeks, compared to 873k in same period for 2.7 (plus another ~45k for Python 3). Fewer, but still not trivial. Fabric's at 8.5%.
  • We're not sure why so many users are still on 2.6...
  • Though it could simply be "it's RHEL/CentOS 6", which includes Python 2.6 as the default
    • Presumably has a very large install base (users of these distros trend conservative) given 7 only came out in 2014
• Supporting 2.6 (in a Python 2+3 world) isn't, technically speaking, significantly more difficult than supporting 2.7.
• "You're doing it wrong" (at folks who could potentially move to 2.7, but haven't yet) always feels like a shitty argument to use to justify making maintainers' lives easier, when the thing under discussion is comparatively easy to keep doing.
  • The key here is that it's becoming rapidly less easy, due to many other packages dropping support...

Possible way forward

• Add a DeprecationWarning (as does pip, cryptography, etc) stating that Invoke 1.0 (maybe 2.0, which is likely to not be years after 1.0) will drop Python 2.6 support. Include a link to this ticket & "please explain why you're still using 2.6".
• Wait a while, see if any other good reasons pop out of that, that aren't already in the counter-arguments list.
• Double-check the latest download stats from PyPI re: % Python 2.6 use (generally and for invoke/fabric/paramiko specifically)
• Make the call re: whether the balance tips enough towards ripping off the band-aid.

[timt1961]

I'm still stuck on python 2.6; have a lot of custom software, that interfaces with rpms on an older set of RHEL6.0 machines. No chance to upgrade in the short term. (We're in a very controlled environment.)

On the other hand, the version of Fabric I'm using works fine for now

[bitprophet]

Thanks @timt1961. Yea, Fabric 1.x is unlikely to do this; even if newer Paramiko versions that Fab 1 is API compatible with, drop 2.6 support sometime, you'll always be able to just pin Paramiko to the previous version.

[bitprophet]

Annoyed that I didn't write down HOW I got the numbers for this. (Also feel like there's at least one other similar discussion around my projects, involving either @dstufft or @alex, but cannot find it...)

What I did in the past and also just now, is:

• Remembered that I'd bookmarked this distutils-sig post which links to these awesome Google BigTable data exports managed by the PyPI folks
• Found the queries I'd run when I last looked at that stuff, and saved one and made it public here (note: seems to require a valid Google login)
• Reran for Paramiko for the last 2 weeks (for consistency with my previously reported numbers)
• Exported to gdocs, here

To reiterate, the LAST time I looked at the Paramiko download data (assuming I did it similarly), we saw:

• Python 2.6 was only 6% of total downloads: 60k downloads over 2 weeks
  • Implying the grant total was thus about 1000k / 1MM
• Python 2.7 was 873k in same period, which would've been 87%
• plus another ~45k for Python 3, or 4.5%

THIS time, going by the above BigData/Sheets links:

• Python 2.6 is down to 4.5%
• Python 2.7 is up to ~89%
• Python 3 is up to ~5.5%
• Unknown/unreported (IIRC this implies pip versions too old to report this data, implying older Python versions, or direct downloads not using pip) is about 1% (FTR, it's like 5-6% for Invoke...huh.)

Sadly I can't tell whether that previous recording was in June around ticket create time, or later on, but either way the story hasn't changed much - 2.6 usage continues to drop, but is still nontrivial, insofar as it's still some thousands of users (even if you assume the numbers are highly inflated due to reinstalls, test runs, etc).

But it's under 5% now. And I need to check out Invoke itself too, of course, though its users are far more likely to skew bleeding edge than Paramiko/Fabric.

[bitprophet]

Invoke itself: https://docs.google.com/spreadsheets/d/1yptr9lEzog0N6fS-AFRFkRVh1hvIxvjUKmWjLA6hXq8/edit?usp=sharing

• Python 2.6 is only 0.2%!
• Though 'unreported' is 6%.
  • So that could be folks on Python 2.6 with old pips.
  • Unlike Paramiko/Fabric, at least, it's super unlikely to be people on Python 2.5 as we're not compatible with it
• Python 2.7 is 58%
• Python 3.5 is 28%, Python 3.4 is 11%, and Python 3.3 is nearly 1%, meaning Python 3 overall is just about 40%! Not bad!

So, re: current early adopters, clearly we could drop 2.6. The question is, how many folks using Fabric and/or Paramiko would find dropping 2.6 to be a major barrier to adopting Fabric 2? (answer is obviously, "up to about 5%").

Still...now that eg Sphinx has dropped Python 2.6 support, it's getting that much harder to justify keeping it around for Fabric 2 and its friends. I think Sphinx is THE largest/most-used dependency of my projects so far to make its own cut here. So my "it's just as easy to support 2.6!" argument is eroded a lot by that. Sad trombone.

[dstufft]

FWIW as of pip 9 and uh, setuptools I'm not sure but let's say the latest one you can add python_requires=">=2.7,!=3.0.*,!=3.1.*,!=3.2.*" to your setup.py and pip 9+ will ignore that file completely if the current version of Python doesn't match the version specifier. You need to make sure you're using the latest setuptools when creating the sdist though.

[dstufft]

To be clear, older versions of pip will still happily install it by default, but newer versions of pip will not. This eliminates (for newer pips) the "I dropped support for Python X.Y but pip install foo on Python X.Y now does the wrong thing" problem.

[bitprophet]

@dstufft - huh, neat! What exactly happens in that case, does pip print an error or just pretend that "uh I cannot find yourpackage==1.2.3"?

[dstufft]

@bitprophet the latter.

[ivoflipse]

Also note that some people use conda: https://anaconda.org/search?q=invoke though ~17k+ is just a drop in the bucket I guess :-)

[bitprophet]

@ivoflipse is Conda not counted in the PyPI stats? (I don't remember exactly how it works re: package grabbing...I guess if it's not using any pip related code, then it's not?)

For Paramiko 17k is a drop in the bucket, for Invoke it'd be like another 50+% increase 😀

[dstufft]

@bitprophet Conda ~= apt/rpm/etc. It is it's own format and they have their own repositories.

[ivoflipse]

This is for invoke only, but a lot of the download will be for our CI since we use it to abstract away running linting/tests.

As for conda, it is indeed completely separate.

[bitprophet]

More arguments for dropping 2.6: the EOL notices from various packages (😁 ) and now, specifically, the fact that readme_renderer doesn't seem to work under 2.6, at least on Travis. Is it a game-ender? No. But every time I need to fuck around with special-case crap like this, it's time I am not spending doing actual useful work. Wears a man down.

[bitprophet]

Prompted by the soon-to-be-linked ticket over in Paramiko land, I reran the numbers for Invoke and as usual 2.6 has dropped more, though is still not as negligible as it is for Paramiko et al.

Spreadsheet: https://docs.google.com/spreadsheets/d/18ks_jPkw7EvZfJW4esUYOr_UwscdIlti-ftBZ3Rpr5M/edit#gid=853072060

Basic results rephrased here: 'unknown' is down to 3.3% from ~6% 9 months ago, and explicit Python 2.6 is down to 0.01% (a whole 16 downloads in the last 2 weeks!). So we're well below 5%, if not at the sub-1% reported on Paramiko (again see to-be-linked ticket.)

It's also 9 months later for me personally and I'm continuing to feel like yea, maybe it is time to get out the axe. I don't actually have time to do said axing right now but it means I'm that much more amenable to doing it before Invoke 1 / Fabric 2 / Paramiko 3.

[dstufft]

Death to 2.6!

[ivoflipse]

It's not like the current release is dysfunctional, Python 2.6 simply wouldn't gain any new features.

[bitprophet]

Got the axe out

Only new fun stuff I found to easily add were set literals/comprehensions & optional string format specifiers ({} vs {0}). None of the spots where we use dicts seemed like good candidates for dict comprehensions.

If anyone has other favorite 2.7/3.x-only upgrades, let me know what I missed ;)