Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate bdist_dumb, bdist_rpm, bdist_dmg, bdist_msi, bdist_wininst #1988

Open
2 of 5 tasks
hugovk opened this issue Feb 9, 2020 · 9 comments
Open
2 of 5 tasks

Deprecate bdist_dumb, bdist_rpm, bdist_dmg, bdist_msi, bdist_wininst #1988

hugovk opened this issue Feb 9, 2020 · 9 comments

Comments

@hugovk
Copy link
Contributor

hugovk commented Feb 9, 2020

Warehouse is planning on removing upload support for bdist_dumb, bdist_rpm, bdist_dmg, bdist_msi, and bdist_wininst:

CPython's distutils deprecated bdist_wininst in 3.8:

And is planning on removing bdist_wininst in 3.9:

In this latter discussion, @pganssle said:

I think the right thing to do here would be to propose removing this [bdist_wininst] in setuptools, and drop the idea of removing it in distutils. The same goes for bdist_rpm and all the other weird non-wheel binary distributions.


What's necessary to deprecate these in setuptools?

PR #1764 deprecates bdist_egg. A similar pattern could be followed to add warnings in run() in bdist_rpm.py and bdist_wininst.py. Does this sound right?

But I'm less clear on bdist_dumb, bdist_dmg and bdist_msi. Are these handled by setuptools, and if so, where?


@pganssle
Copy link
Member

Thanks for kicking off this process, Hugo!

I think we can follow the road map from setup.py test and setup.py upload/register:

  1. Add deprecation warnings for 9-12 months (with tests).
  2. Document the deprecation.
  3. During removal time, add new sub-commands that actively raise errors (rather than simply removing the commands from setuptools).

But I'm less clear on bdist_dumb, bdist_dmg and bdist_msi. Are these handled by setuptools, and if so, where?

setuptools is a wrapper around distutils (and hopefully eventually distutils will move into setuptools), so anything that's setuptools doesn't modify / monkey-patch just is passed through to distutils. The two options we have for deprecating bdist_dumb et al would be creating our own sub-command that raises a deprecation warning or monkey-patching the relevant commands in distutils. Normally monkey-patching is somewhat abhorrent to me, but in this case I'm inclined towards monkey-patching so that the warnings hit the greatest number of people (e.g. people who for some reason or another are subclassing distutils.command.dumb or whatever).

It would probably be nice for us to prepare a blog post or some other similarly "loud" announcement of these forthcoming removals. Probably we can do that during the deprecation period, though.

@hugovk hugovk mentioned this issue Mar 22, 2020
2 tasks
@Cimbali
Copy link

Cimbali commented Apr 10, 2020

Should these commands be removed from setuptools/distutils? I understand that these formats are not going to be supported on PyPI anymore − however this is based on PyPI download statistics.

Arguably, building binary releases in non-wheel binary formats is still being used for different distribution channels.

In fact, I find centralising all the build meta-data in setup.cfg to build on all platforms with different commands extremely useful, rather than needing different processes, configurations, etc. for every targeted release platform.

bdist_wininst seemed to have additional implementation issues from what I read in the linked discussions. The other formats (rpm, dmg, msi) are not discussed. What’s the rationale for deprecating those?

@hugovk
Copy link
Contributor Author

hugovk commented Apr 13, 2020

PEP 527 has rationale for removing those formats from PyPI, and not only based on downloads.

As of today, PyPI has now fully implemented PEP 527:

@Cimbali
Copy link

Cimbali commented Apr 13, 2020

The rationale of PEP 527 is:

  • bdist_dumb is not usable
  • maintaining the non-wheel binary formats on PyPI is undesirable
    • as it has a significant bandwidth cost,
    • and because other distribution channels do it better.
  • the impact of removing such formats from PyPI should be low, because
    • most of these are barely used,
    • the only format significant download traffic on these formats (bdist_wininst) actually is due at 90% from mirrorring

and then a discussion of non-binary format and extensions.

Since the commands associated with these formats are also used to build packages that are not shipped through PyPI, it makes the impact of such a deprecating these commands harder to gauge than centralized download statistics. I see nothing in there that straightforwardly implicates commands should be removed from setuptools, except for bdist_dumb.


In the end, I’m only asking to be enlightened on the pros and cons of removing the bdist_rpm, bdist_dmg, bdist_msi, bdist_wininst commands from setuptools.

  • Are they a significant maintenance overhead? Is there another cost in keeping them around?

  • Am I missing the point and is setuptools only meant to be an interface to/from PyPI? (The host of other commands such as build_sphinx or flake8 would indicate not.)

  • Is there available data on how widely they are (un)used?

  • Is it likely useful alternatives can be found?

    For example to build rpms, the only alternative short of writing spec files by hand is pyp2rpm, which is both much less equipped (can’t even specify dependencies outside of other python packages) and apparently in need of new maintainers. I haven’t had the time to take a deeper look at pynsist yet, but that seems an option to replace bdist_wininst.

  • etc.

@jaraco
Copy link
Member

jaraco commented Sep 25, 2021

* Am I missing the point and is setuptools only meant to be an interface to/from PyPI? (The host of other commands such as `build_sphinx` or `flake8` would indicate not.)

The primary motivation here is that Setuptools should no longer be a general-purpose packaging tool, but should focus on being a best-in-class build backend for the Python Packaging ecosystem, based on the standards set forth by the PyPA. To that end, Setuptools is seeking to deprecate setup.py <anything> and instead support the PEP 517 build backends, which only target building wheels. So yes, I would expect to discourage/deprecate usage of distutils commands such as build_sphinx and flake8.

The essential problem with supporting bdist_rpm is that it requires every Python project to adopt Setuptools to have RPM support.

Instead, probably what needs to happen is for someone to build a tool to convert wheels to RPMs. Then, that tool could generate an RPM from any wheel, including those built by other backends. Maybe that's what pyp2rpm does.

@Cimbali
Copy link

Cimbali commented Sep 25, 2021

Thanks for the answer @jaraco

Setuptools should no longer be a general-purpose packaging tool

That makes a lot more sense than quoting PEP 527. I’m not keen on that direction, as I really valued the extensible single-entry-point interface that setuptools provided, but so be it.

Note that to your point about bdist_rpm, this deprecation doesn’t make it more possible to have RPM support without setuptool’s bdist_rpm. We go from 3 options: do it manually / use setuptools / roll your own tool, to just 2: do it manually / roll your own tool.

Have you also considered what this implies for packages that extend setuptools to provide packaging support, e.g. cx_Freeze? These are doing the work you’re no longer wanting to do with bdist_msi etc. so I think they are valuable people to talk to and give advance notice of what will happen when the deprecations are enacted.

  • Will they still be able to extend setup() with their own commands? Or will their whole process have to be changed in the future to build from wheels?

If the extensibility remains, such packages will likely have to duplicate all the bdist_msi and similar code they’re extending now to take ownership of maintaining that. That’s a ton of work that probably shouldn’t be asked if in the end it’s useless and they should write a wheel-to-something converter from scratch instead.

jaraco added a commit that referenced this issue Oct 22, 2021
Cimbali added a commit to Cimbali/pympress that referenced this issue Nov 9, 2021
setup.py bdist_rpm is being (slowly?) deprecated, see pypa/setuptools#1988
Also call rpmlint
smbct pushed a commit to smbct/pympress that referenced this issue Aug 4, 2023
setup.py bdist_rpm is being (slowly?) deprecated, see pypa/setuptools#1988
Also call rpmlint
@abravalheri
Copy link
Contributor

It's been 2 years since we have start issuing deprecation warnings for bdist_rpm.

Should we remove it for good in November?

CC @jaraco

@twmr
Copy link
Contributor

twmr commented Nov 28, 2023

What are the plans for setuptools once the bdist_* commands are removed from the code-base? Does it make sense that versions of the bdist_* commands are maintained by the community outside of setuptools repo? I mean will setuptools keep providing the infrastructure for such out-of-tree maintained bdist commands?

@abravalheri
Copy link
Contributor

Hi @thisch, the plan in setuptools for the last (many) years is to phase out any CLI tool capabilities. This means that the intended public API is mostly related to standardised build backend hooks and the entry_points meant for plugins. We haven't fully discussed the future of the public API, but the items I mentioned are what I would call the planned A-tier support.

adelton added a commit to adelton/rpm2swidtag that referenced this issue Feb 17, 2024
Addressing
$ make spec
python3 setup.py bdist_rpm --spec-only --install-script=bdist_rpm/install_script
running bdist_rpm
/usr/lib/python3.12/site-packages/setuptools/_distutils/dist.py:988: SetuptoolsDeprecationWarning: Deprecated command
!!

        ********************************************************************************
        bdist_rpm is deprecated and will be removed in a future version.
        Use bdist_wheel (wheel packages) instead.

        This deprecation is overdue, please update your project and remove deprecated
        calls to avoid build errors in the future.

        See pypa/setuptools#1988 for details.
        ********************************************************************************

!!
adelton added a commit to adelton/rpm2swidtag that referenced this issue Feb 17, 2024
given deprecation of bdist_rpm in pypa/setuptools#1988
does not provide any solid guidance about replacements.

We just use a .spec file template in the end. But it does not seem
to work on CentOS Stream 8 and 9, so we have to stop testing on those.

We wanted to use pyproject.toml for everything.
But tool.setuptools.data-files does not respect absolute path
(unlike setup.py's data_files) so that has to be done with installs
in the .spec; and py-modules fails with
    configuration error: `tool.setuptools.py-modules[0]` must be python-module-name
    GIVEN VALUE:
        "dnf-plugins.swidtags"

    OFFENDING RULE: 'format'

    DEFINITION:
        {
            "type": "string",
            "format": "python-module-name"
        }
so we have to put those in setup.cfg.

https://gregoryszorc.com/blog/2023/10/30/my-user-experience-porting-off-setup.py/
adelton added a commit to adelton/rpm2swidtag that referenced this issue Feb 17, 2024
to get rid of

    $ make spec
    python3 setup.py bdist_rpm --spec-only --install-script=bdist_rpm/install_script
    running bdist_rpm
    /usr/lib/python3.12/site-packages/setuptools/_distutils/dist.py:988: SetuptoolsDeprecationWarning: Deprecated command
    !!

            ********************************************************************************
            bdist_rpm is deprecated and will be removed in a future version.
            Use bdist_wheel (wheel packages) instead.

            This deprecation is overdue, please update your project and remove deprecated
            calls to avoid build errors in the future.

            See pypa/setuptools#1988 for details.
            ********************************************************************************

    !!

Given pypa/setuptools#1988 deprecates bdist_rpm
but does not provide any solid guidance about replacements, we just use
a .spec file template in the end. But the necessary dependencies do not
seem to be present / work on CentOS Stream 8 and 9, so we have to stop
testing on those.

We wanted to use pyproject.toml for everything.
But tool.setuptools.data-files does not respect absolute path
(unlike setup.py's data_files) so that has to be done with installs
in the .spec; and py-modules fails with
    configuration error: `tool.setuptools.py-modules[0]` must be python-module-name
    GIVEN VALUE:
        "dnf-plugins.swidtags"

    OFFENDING RULE: 'format'

    DEFINITION:
        {
            "type": "string",
            "format": "python-module-name"
        }
so we have to put those in setup.cfg.

https://gregoryszorc.com/blog/2023/10/30/my-user-experience-porting-off-setup.py/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants