Deprecate .egg uploads #10653
Comments
|
My (biased) industry exposure is that the only major place where eggs come up at this point (and where developers stumble over them) is PySpark. Specifically the spark ecosystem still prominently mentions eggs as the way to attempt to fan out dependencies. At some point I'm sure I investigated why they still have not caught up, but yeah that's where I see eggs ever mentioned personally, with big data developers who find that and don't know wheels replaced them a long time ago. |
|
Additional data points:
In rolling this out, it seems reasonable to add a notification email to the uploader of any bdist_egg at upload time warning them and pointing to some documentation and a timeline. |
|
Also, to be clear: I'm +1 to deprecating egg uploads. The Removal Process section of PEP 527 shows prior art for this kind of thing in the past. I'm not sure if we would want to go so far as the "disable with exception" route, especially if we are notifying uploaders. |
|
Just for reference, there's a Spark issue to add wheel support, as well as another proposal, but it was automatically closed many years ago and no one ever implemented it. Based on their docs and the comments on those issues, it seems the community may have moved on from the native deployment mechanism (which included the egg support), with virtualenv support and the other deployment mechanisms mentioned on the above-linked docs page being seen as potentially better alternatives. |
|
Based on https://twitter.com/yenzenz/status/1486416369340010508?t=Fp1VvBzrztarwCX0pmgIyw&s=19 looks like the Plone community still needs the ability to upload eggs for a while, so it might make sense to give existing projects a longer period to migrate, but start blocking this for new projects in the short term after a deprecation period. |
|
@jensens Thanks for the information! If Plone 6 is released this year with wheel support built in, and Plone 5.2.x gains support through buildout v3 within the same general timeframe, would something like one year of PyPI upload deprecation (i.e. Jan/Feb 2023) be sufficient? All this proposal would do is prevent uploading new egg distributions for future versions, it wouldn't stop your users from still getting existing egg-only updates indefinitely. So as long as at least one version was released with wheel support, as an egg before the end of the deprecation period, your users should presumably be able to upgrade to that version, and then to any future versions. Certainly a possible two-step upgrade process for users not keeping up with updates (as your security policy states they should) is not quite optimal, but it should still work, right (and since you maintain your own tooling, you could smooth out any UX kinks)? |
|
I think the sketched plan works for the Plone community. We stick to the old stuff too long and we are aware of it. I fear we are responsible for a significant part of the eggs. Anyway, I would like to get an opinion from our release managers @mauritsvanrees and @esteele here as well. |
|
Hello from one of the Plone release managers. I don't remember the last time I uploaded an egg, maybe I never did this. I always upload the source distribution, and the last few years a wheel. But the packages I handle are pure Python, no C code. For Plone 5.2 (the current stable release), I expect to give about two more years of security support, so until early 2024. The happy path would be that Plone 5.2 users could still use zc.buildout 2. This does not use pip, but a heavily edited own copy of easy_install, so I usually couple this with setuptools 42 to avoid getting deprecation warnings. And Plone 5.2 can still run on Python 2.7, so ideally that would still work as well. Yes, old. Also, just to be clear: Buildout itself can already install wheels since version 2.9 or so, which is several years old. The Plone packages do not care whether a package is installed from an sdist, an egg, or a wheel. So no action needed there. Summary: a deprecation period of about a year should be fine for us. |
Not that I am aware of. |
We stopped uploading eggs many years ago. The last one I could find using a quick search was from 2018. Nowadays we use sdist for Python-only packages and additionally wheels if there is C code involved. From Zope perspective I see no problem in removing support for uploading eggs. |
|
What’s the next steps to move forward on this? |
|
Sounds like it's for PyPI start raising a warning when an Egg is uploaded, and then after some period of time (say, 6 months or a year), drop support. @jensens @mauritsvanrees What's Plone's status on this? Skimming your site, it looks like Plone 6 is in rc2, so I presume release is imminent. Has Plone 5 moved to buildout 3? When did that happen? |
Something like PEP 527#Removal Process / #6792? |
|
|
Plone 6.0.0 final was released today. :-) It is using either pip, or Buildout 3. Plone 5.2 is using Buildout 2.13 by default, and this can already install wheels. (You can use pip too.) Older Plone versions are no longer supported. From the comments above, the only thing that would change is disallowing uploads of new eggs. That should be no problem anyway. |
|
We're going to move ahead with implementing the block. @woodruffw would like to do the implementation. This will include a blog for notice and emails when egg upload are attempted. |
|
I think we should probably do a PEP for this, like https://peps.python.org/pep-0527/, as @hugovk mentioned above. |
|
@woodruffw wanna write a PEP? |
I think I've sniped myself into it 🙂 |
|
Cross-referencing: I've opened a draft PEP here: python/peps#3161 |
How minuscule is it now? |
In all of May 2023: 581 eggs were uploaded among the 236,261 total distribution files, representing 0.2% of files uploaded to PyPI. |
|
PEP 715 was accepted, so this is unblocked! A list of the tasks that need doing:
I'm not 100% sure how mass email blasts work with PyPI's infra, but I'll start work on the second two while I figure out the first! |
|
Started in on this: https://github.com/pypi/warehouse/compare/pep_715
Got beat up by tests.
… On Jun 25, 2023, at 4:43 PM, William Woodruff ***@***.***> wrote:
PEP 715 <https://peps.python.org/pep-0715/> was accepted, so this is unblocked!
A list of the tasks that need doing:
Send a mass email to every maintainer of a package that has uploaded an egg distribution since 2023-01-01;
Update the upload endpoint to send an email to each maintainer whenever an egg distribution is uploaded (for the next month);
Draft a post for the PyPI blog announcing the deprecation period.
I'm not 100% sure how mass email blasts work with PyPI's infra, but I'll start work on the second two while I figure out the first!
—
Reply to this email directly, view it on GitHub <#10653 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJFFQFUI3MRT547IXMZFP3XNCPHBANCNFSM5M3VQ4WQ>.
You are receiving this because you commented.
|
I can take a look at tests; will send a PR to your branch. |
|
#14017 for the initial rollout is now ready for review |
|
The initial notification to all maintainers of projects that have uploaded eggs since Jan 1, 2023 has now been sent. We'll need a PR to finalize the deprecation for merge on August 1, 2023. |
|
Bump on this, we are ~2 weeks from the Aug 1 deadline. @woodruffw are you planning to produce that PR? |
I am! I can spend some time on that today. |
|
Opened #14118 for the above. |
Currently PyPI accepts uploads for two distribution types: source distributions, and built distributions. PyPI accepts two built distribution types: wheels and eggs:
https://github.com/pypa/warehouse/blob/7732f2fb9eb0b1da032731d926a094f5f5d1488f/warehouse/forklift/legacy.py#L542
Eggs have never been widely used and have been far outpaced by wheels in recent years:
Some additional details:
pipdeprecated the--eggcommand in 2016: Deprecate --egg pypa/pip#1749setuptoolsdeprecated thebdist_eggcommand in Deprecated Eggsecutable Scripts pypa/setuptools#1764buildonly supports wheels for built distributions: https://pypa-build.readthedocs.io/en/latest/#python--m-build---wheelThe text was updated successfully, but these errors were encountered: