Can wheels install arbitrary files #576
Comments
|
No. See the wheel spec. All files in a wheel are installed to one of the distribution-specified purelib/platlib/headers/scripts/data locations. |
|
Imagine I'm writing a new project in Python which wants to install manpages ( How would I do that using the modern packaging systems? |
|
As that's integrating with the Unix system, and isn't expected to work on (for example) Windows, I'd say you should be building an RPM or a DEB for it, not a wheel (which is a cross-platform format for Python libraries). I don't know much about Unix packaging, so I can't say much more than that, though, I'm afraid. |
|
Sorry, I should have been clear. distutils is deprecated already and will be removed in 3.12, and setuptool's |
|
Python's packaging ecosystem isn't designed to install Unix system files, that's what I'm trying to say. It never was - I'm not 100% sure how distutils/setuptools is relevant here (they aren't designed for installing Unix system files either). Maybe I'm missing something, and historically people somehow used Python tools like distutils to install Unix system files. I don't think that was ever intended/supported use (I've heard a lot of Linux distro maintainers complaining that people should use the Linux management tools, not Python, to alter system-managed files) so there's no "modern way" of doing this, and if it was possible in the past then I guess sorry, it's no longer supported... |
|
Interesting, thanks. FWIW, there's plenty of software out there which subclasses the |
Running |
Interesting. Presumably that software would need to be installed using So yes, I'd strongly recommend switching to building an rpm/deb for applications like this. |
|
Packaging RPM/DEB is orthogonal to this issue. How would one generate the rpm/deb in the first place? The typical process is:
If I've a 100% python package that also wants to install an init script, is the only solution to use something like Meson/cmake/autotools and not use the Python packaging infrastructure at all? |
|
To clarify about the root thing: in distribution-land the packager runs |
With config files and scripts specific to the target distro. |
|
Distutils et al. allow you to install system files to /usr on a linux system -- this is fine for most purposes. Indeed, that is then staged and distutils' You don't need to hack or override install commands for that! The distutils/setuptools However, IIRC this only works using
Actually, distro maintainers complain that Given that
Yes. For example, you can use Meson, and The final question is whether you want to register the package as installed to python's language-specific package manager, that is to say registering it to If you're just installing a CLI tool then of course you may not care if there isn't a dist-info when installed with |
To be honest, neither Meson nor autotools tries to build deb or rpm themselves either, they just try to support generic system installation (and they kind of have to because they are multi-language build systems). CMake does try to build deb or rpm itself, which personally I think is a bit of madness and isn't desirable from either a generic multi-language buildsystem perspective or a "try to package it for a linux distro" perspective. The Python packaging tools started off closer to Meson/autotools than to CMake (technically there is bdist_rpm, but has anyone ever used it? how many people know it exists? and there is no bdist_deb), and have moved on to only providing wheels which install either pure python, compiled python, or python data, and don't get involved in anything that isn't strictly python. Python is officially out of the business of doing "integration", not "OS formats". |
|
Ross and I both work on Yocto Project and I recently contributed the move to wheels and pyproject.toml/PEP-517 build-backends. Canonical and Red Hat have a lot of legacy tools still depending on the distutils behavior (e.g. ufw). Given how much notice they have had to upgrade to setuptools/setup.py we shall see what happens when they are then asked to use meson or other tools to package their tools since Python will no longer do it. There are also legacy tools in the git.kernel.org tree. For the distro maintainer workflow, we are at the mercy of the new packaging (I understand the reasons and am a fan) and the upstream projects we do not maintain, but need to package. There probably is no perfect world. |
Following a good discussion with PyPA upstream[1] the migration of the setuptools3.bbclass to use bdist_wheel+pip turns out to be more complex than thought. Essentially, we're midway through a lot of changes: the future of Python packaging is wheels and pip, but those by design are not as flexible as traditional distutils and setup.py. Specifically, with traditional distutils the package can implement its own install task and write arbitrary files (such as init scripts). With wheels this is explicity impossible, so packages that do this cannot use the new setuptools class and must continue to use the build/install tasks as before. This class is the old setuptools behaviour, bought back. However, as distutils and the setuptools install task are both deprecated and will soon be removed entirely, any users of this class should be moving to an alternative build tool, be it a modern Python tool which works with wheels, or a non-Pythonic tool such as Meson. [1] pypa/packaging-problems#576 Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Khem Raj <raj.khem@gmail.com>
Following a good discussion with PyPA upstream[1] the migration of the setuptools3.bbclass to use bdist_wheel+pip turns out to be more complex than thought. Essentially, we're midway through a lot of changes: the future of Python packaging is wheels and pip, but those by design are not as flexible as traditional distutils and setup.py. Specifically, with traditional distutils the package can implement its own install task and write arbitrary files (such as init scripts). With wheels this is explicity impossible, so packages that do this cannot use the new setuptools class and must continue to use the build/install tasks as before. This class is the old setuptools behaviour, bought back. However, as distutils and the setuptools install task are both deprecated and will soon be removed entirely, any users of this class should be moving to an alternative build tool, be it a modern Python tool which works with wheels, or a non-Pythonic tool such as Meson. [1] pypa/packaging-problems#576 Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Following a good discussion with PyPA upstream[1] the migration of the setuptools3.bbclass to use bdist_wheel+pip turns out to be more complex than thought. Essentially, we're midway through a lot of changes: the future of Python packaging is wheels and pip, but those by design are not as flexible as traditional distutils and setup.py. Specifically, with traditional distutils the package can implement its own install task and write arbitrary files (such as init scripts). With wheels this is explicity impossible, so packages that do this cannot use the new setuptools class and must continue to use the build/install tasks as before. This class is the old setuptools behaviour, bought back. However, as distutils and the setuptools install task are both deprecated and will soon be removed entirely, any users of this class should be moving to an alternative build tool, be it a modern Python tool which works with wheels, or a non-Pythonic tool such as Meson. [1] pypa/packaging-problems#576 (From OE-Core rev: ec157ece77c5dd1e05d08127228fb3ad98e3d667) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Following a good discussion with PyPA upstream[1] the migration of the setuptools3.bbclass to use bdist_wheel+pip turns out to be more complex than thought. Essentially, we're midway through a lot of changes: the future of Python packaging is wheels and pip, but those by design are not as flexible as traditional distutils and setup.py. Specifically, with traditional distutils the package can implement its own install task and write arbitrary files (such as init scripts). With wheels this is explicity impossible, so packages that do this cannot use the new setuptools class and must continue to use the build/install tasks as before. This class is the old setuptools behaviour, bought back. However, as distutils and the setuptools install task are both deprecated and will soon be removed entirely, any users of this class should be moving to an alternative build tool, be it a modern Python tool which works with wheels, or a non-Pythonic tool such as Meson. [1] pypa/packaging-problems#576 Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Following a good discussion with PyPA upstream[1] the migration of the setuptools3.bbclass to use bdist_wheel+pip turns out to be more complex than thought. Essentially, we're midway through a lot of changes: the future of Python packaging is wheels and pip, but those by design are not as flexible as traditional distutils and setup.py. Specifically, with traditional distutils the package can implement its own install task and write arbitrary files (such as init scripts). With wheels this is explicity impossible, so packages that do this cannot use the new setuptools class and must continue to use the build/install tasks as before. This class is the old setuptools behaviour, bought back. However, as distutils and the setuptools install task are both deprecated and will soon be removed entirely, any users of this class should be moving to an alternative build tool, be it a modern Python tool which works with wheels, or a non-Pythonic tool such as Meson. [1] pypa/packaging-problems#576 (From OE-Core rev: bb58ec20a917f8f7f815c465eed5d0b953bf3bfb) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Following a good discussion with PyPA upstream[1] the migration of the setuptools3.bbclass to use bdist_wheel+pip turns out to be more complex than thought. Essentially, we're midway through a lot of changes: the future of Python packaging is wheels and pip, but those by design are not as flexible as traditional distutils and setup.py. Specifically, with traditional distutils the package can implement its own install task and write arbitrary files (such as init scripts). With wheels this is explicity impossible, so packages that do this cannot use the new setuptools class and must continue to use the build/install tasks as before. This class is the old setuptools behaviour, bought back. However, as distutils and the setuptools install task are both deprecated and will soon be removed entirely, any users of this class should be moving to an alternative build tool, be it a modern Python tool which works with wheels, or a non-Pythonic tool such as Meson. [1] pypa/packaging-problems#576 Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Following a good discussion with PyPA upstream[1] the migration of the setuptools3.bbclass to use bdist_wheel+pip turns out to be more complex than thought. Essentially, we're midway through a lot of changes: the future of Python packaging is wheels and pip, but those by design are not as flexible as traditional distutils and setup.py. Specifically, with traditional distutils the package can implement its own install task and write arbitrary files (such as init scripts). With wheels this is explicity impossible, so packages that do this cannot use the new setuptools class and must continue to use the build/install tasks as before. This class is the old setuptools behaviour, bought back. However, as distutils and the setuptools install task are both deprecated and will soon be removed entirely, any users of this class should be moving to an alternative build tool, be it a modern Python tool which works with wheels, or a non-Pythonic tool such as Meson. [1] pypa/packaging-problems#576 (From OE-Core rev: 341d2b35986e48e4954c591be8bc037a5557452a) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Wait, I'm confused. What exactly are you saying works with setuptools/distutils directly, but not with wheels? |
|
Files with absolute paths, such as Also extremely useful for /etc, or more precisely the configurable As mentioned by the author of this ticket:
There is:
None of these reliably work if your view of the installation environment is "python environment only, nothing outside of sys.prefix". While I am sympathetic to PyPA's desire to "get out of that game" and leave it to packaging tools such as meson, autotools, cmake, or Makefiles, it remains true that users who need this did in fact have something that served their needs in ... Also absolute paths are a hack, because what you actually want is, for example, Meson's ability to do this: And then you will be able to do a user install of $XDG_DATA_HOME compatible files to ~/.local/share/, while putting python modules into idk, Of course not everyone actually needs desktop files, and For example building a wheel with meson, its good support for cross-language extension compiling, and a PEP517 backend, then having the wheel not try to do desktop files, and handle scripts via entry point metadata that adapts to a different install layout than the one it was built for. |
Problem description
Can a wheel install an arbitrary file? For example,
/lib/systemd/system/foo.serviceon Linux.The text was updated successfully, but these errors were encountered: