Data files copied incorrectly

[WhyNotHugo]

Hi! I maintan caffeine-ng. When installing via setup.py, everything installs as expected:

python setup.py install --root="$pkgdir" --optimize=1 --skip-build

Installs, amongst other files:

/usr/share/icons/hicolor/16x16/apps/caffeine.png

But when using installer:

 python -m installer --destdir="$pkgdir" dist/*.whl

The files are installed into the wrong path, the example from above ends up in:

/usr/lib/python3.10/site-packages/usr/share/icons/hicolor/16x16/apps/caffeine.png

This is my relevant code: https://codeberg.org/WhyNotHugo/caffeine-ng/src/commit/e351249f3913e3f1354fba768803063597972125/setup.py#L9-L28

I'm not entirely sure what's wrong. Is this usage unsupported by installer? Is it deprecated, or pending implementation? Or have I found a bug?

[eli-schwartz]

This is not about installer vs. setuptools, but rather, it's more generally about setup.py install vs. using wheels.

The wheel spec doesn't account for files installed outside of prefix (that is, /usr) so you cannot install to /etc. It also doesn't handle absolute paths as part of prefix, even if the prefix is the same, which may or may not matter depending on whether you would want to install data to /usr even when python is not in /usr.

python setup.py install doesn't use wheels, and has unixy handling of data files.

It's also deprecated, largely because PyPA wants to reduce the scope of "python-specific build systems" and just consolidate on wheels to handle the problem of python modules, not all the other crazy things multi-language projects with deep system integrations get up to. :p

I talked about this a bit at pypa/packaging-problems#576 (comment)

[WhyNotHugo]

I've been considering that meson might be a better fit for this kind of usage (e.g.: a regular desktop application which include a desktop entry, icons, systemd service, and other files in /usr/share).

Based on your reply, it sounds that this is the recommended approach for this kind of applications too, right? I guess that providing wheels for these is also no longer a great idea either.

[uranusjr]

Yes, Meson would be a better fit for this sort of use cases. Publishing wheels wouldn’t work (and any tool letting a wheel work like this should be considered violating the rules).

[pradyunsg]

I don't have much more to add, but... Yea, this sounds like you'll have to avoid wheels as a whole and use a different more-flexible build mechanism (meson, CMake etc) for handling, at least, the data files. Depending on the project and your preferences, you may also be able to use those to package the project.

[WhyNotHugo]

In this case, it's about caffeine-ng, but this is also something that somewhat affects vdirsyncer, todoman and khal, and probably other desktop applications (which include things like services, shell autocompletions, man pages, etc).

I'll try and experiment with meson a bit, it does seem well suited for the task, and in this particular case, can also deal with non-python dependencies. My initial goal is to have two modes of operation:

• One for distro packagers, where I just install the python module into site-packages and everything else into its proper place.
• Once for end-users, where a virtualenv is created in /usr/share with my module and all its dependencies (so as not to mess up python modules provided by the distribution).

This should be able to compensate for the fact that "simply" installing into a virtualenv via pip would be somewhat unsupported.

I'll try and prototype this a bit and share my findings -- I'm sure others will come asking the same questions for similar scenarios.

[WhyNotHugo]

Thanks for the guidance here, I'll go ahead and close this; there's nothing left to discuss here.

[eli-schwartz]

Based on your reply, it sounds that this is the recommended approach for this kind of applications too, right? I guess that providing wheels for these is also no longer a great idea either.

That's a big fat "maybe".

You might want to support some sort of "degraded" mode where you can install the executable entry point with pip install, but not the deep system integrations. It could be useful to some people, depending on the application and depending on the use case. The question is whether that convenience (as opposed to git clone && meson setup && meson install) is worth the degraded functionality and possible confusion.

e.g. it might be an acceptable tradeoff to not install the man page and a systemd service. It might be an unacceptable tradeoff to not install a desktop file and an icon theme.