Deprecate LegacyVersion/LegacySpecifier

[di]

There's too much confusion about how the LegacyVersion and LegacySpecifier interact with their non-legacy counterparts when mixed. (See #74, #112, #275, #307, #320, and probably more)

In #12 @dstufft said :

I don't think it's something that we're going to be able to realistically deprecate anytime soon.

But that was six years ago. I think the community has been publishing/using non-legacy versions for long enough that we can remove the need for this library to support it.

I propose a deprecation period, where creating a LegacyVersion/LegacySpecifier produces a deprecation warning, followed by a removal, where this would raise an InvalidVersion/InvalidSpecifier exception instead (possibly with some additional details about them being legacy).

[dstufft]

We should probably figure out how many projects this would make uninstallable on PyPI before we decide to do this.

[xavfernandez]

Without necessarily deciding on a deprecation period, simply poping up a Warning when LegacyVersion / LegacySpecifier are used as fallback in packaging.version.parse & SpecifierSet.__init__ would be a less impactful first step.

[brettcannon]

I agree with @dstufft that a quick analysis of the latest releases of projects to see which ones would get excluded would be good. But I will also say that the consistent confusion about LegacyVersion suggests we should deprecate it regardless of the analysis outcome and that the analysis informs more how quickly we can drop it rather than whether we should.

[pradyunsg]

Q: Who's doing the analysis tho?

[di]

I did the analysis (scripts and data are here).

Out of nearly 2M releases on PyPI, less than 0.2% releases have a LegacyVersion (5172 total)

The last release to publish with a LegacyVersion was published on 2015-09-10.

[pradyunsg]

Thanks @di!

I say we can deprecate now (say, in a release this month?) and drop support in early 2021.

[dstufft]

Do you happen to have the spare cycles to do the same thing with specifiers @di? Just using the info that's in the eDB should be good enough.

The concern is that it doesn't surprise me that versions are a minor %, given PyPI has actively rejected non PEP 440 versions for awhile, so anything that has been released recently is incapable of not being a valid PEP 440 version. However, I don't think we do the same validation on dependency specifiers, which means people could be publishing invalid specifiers still to this day.

[di]

@xavfernandez @pradyunsg Regarding a deprecation warning, I'm thinking that we might want to handle this differently when deprecating in this library vs. deprecating in pip.

Here, I'd expect any time a LegacyVersion is initialized to emit a warning, but I think that has the potential to produce a lot of warnings when pip does it's usual things. I think instead in pip we'd want to just emit a warning whenever a user has explicitly constructed a LegacyVersion via a specifier, etc.

But I'm not super familiar w/ pip's internals, so maybe that isn't necessary?

[pradyunsg]

deprecating in this library vs. deprecating in pip

FWIW, I realized after reading your comment that we'd probably want to expose this warning in some form to end users of pip. I was thinking purely in terms of deprecating in this library. :)

But I'm not super familiar w/ pip's internals, so maybe that isn't necessary?

I think we'd be putting this through the warnings module here and, IIUC, you would only see 1 warning get printed regardless of how many class calls (thanks autocorrect) are made unless the warning module's default configuration is changed. And pip doesn't do that, so it'll be fine?

---

don't think we do the same validation on dependency specifiers

Huh. TIL.

I'd argue: if warehouse isn't performing validation on specifiers, maybe we should treat these as two separate deprecations? :)

[di]

I parsed all the specifiers as well (scripts are here, GitHub couldn't handle the CSV file)

Of the nearly 5M requirement strings, only 5719 parsed as including a LegacySpecifier.

In addition, a lot of these look like typos, e.g. plone.app.form>=1.1.8plone.app.vocabularies

[dstufft]

Cool, that strongly suggests that deprecating and ultimately removing these things is not going to cause any major problems.

The only other comment I have is it might make sense to keep the classes around, but simply stop the fallback to the Legacy classes when parsing. That would mean that by default we would fail on legacy versions and specifiers, but if someone is using the API directly, and they have a reason to, they can still interact with legacy versions/specifiers. I don't believe either of these modules really requires much in the way of maintenance, so I don't think it would be a large maintenance burden or anything.

The main reason I could see not to do that, is specifically for the specifier code, is that there are some minor simplifications that could be made to the logic if we completely drop the legacy forms.

[brettcannon]

I say drop the old versions and if people really need them they can either copy the code or rely on an old version of 'packaging'.

[pradyunsg]

I agree with @brettcannon.

So, if no one is opposed, uhm, let's deprecate both the things in packaging this month, scheduling for removal in early 2021?

I'm kinda thinking of 21.0 as a "drop legacy cruft" releases for both pip and packaging, although in this case, we could definitely get away with a shorter deprecation period.

[jimporter]

I suppose I can always handle this another way, but for what it's worth, I use this package primarily to handle version checking of non-Python projects used by Python code. Arguably, this isn't the point of this package and I shouldn't have relied on this package to do this in the first place, but it's nice to have a generic way of dealing with version strings in Python. I'm not sure how many other people out there do something similar, but at least for me, this package isn't solely useful for looking at version strings on PyPI.

I for one would be perfectly happy to see PEP440 and "legacy" versions fully-separated from each other (in fact, I already do this). However, I can see the argument that "legacy" (or "non-Python") version handling doesn't really belong in the Python packaging module. At the risk of inflicting scope creep on another team, would it make sense to pull out the version and specifiers modules into another package that packaging then depends on? Then packaging could solely use PEP440 versions, but other Python projects out there could use both PEP440 and legacy/non-Python versions as needed.

Of course, I understand if the PyPA team doesn't want to add the maintenance burden of creating a new versioning package, but I thought it would be worth suggesting, since I'm sure there are other people out there who want to work with non-PEP440 versions in Python.

[di]

Thanks for sharing @jimporter. I think unfortunately that's a use case that this project isn't able to support.

That said, the license would permit pulled out the LegacyVersion/LegacySpecifier logic into a separate package that handles "generic" versions, with proper attribution. It sounds like there's probably a lot in packaging that you don't need, so this might simplify things for you as well. I think if a third party wanted to do this, there's nothing stopping them, but the packaging maintainers don't have the bandwidth to pursue this.

[jimporter]

@di Ok, thanks for the info. Do you foresee any major changes to the APIs for the PEP440-style versions? At a quick glance, it shouldn't be too difficult to provide an API-compatible package that solely provides "legacy" versioning; then, it'd be easy to use one or the other or both, as needed. In fact, fully-separating PEP440 versions and legacy versions would probably make both implementations a bit simpler: packaging could eliminate a bunch of code, and a legacyversion package would only need stub implementations of a lot of methods (e.g. prerelease handling).

That said, I'm not sure yet what I'll do here; maybe for my purposes, it'd be better to parse versions differently from how LegacyVersion works today. I really didn't think overly hard about what the exact behavior of LegacyVersion would be when applied to non-Python projects, and now seems like a good time for me to revisit that decision. :)

[dstufft]

It's unlikely there are any major changes to the Version API, except perhaps additional methods or properties to allow better introspection of them.

[di]

Another example: #341

[pradyunsg]

3 months is enough time, yea? Let's deprecate stuff in master now, and drop them in a 21.0 release. I like the idea of starting 2021 with a nicer situation.

[gaborbernat]

Out of nearly 2M releases on PyPI, less than 0.2% releases have a LegacyVersion (5172 total)

https://pypi.org/project/setuptools-scm generated local versions parse as a legacy, see #356. So this change might impact local development, though not published packages.