Wheel ABI tag heuristics producing `py3-none` again on Arch Linux

[rgommers]

To reproduce this, install meson master (currently at commit 83d18d137) and meson-python main (currently at commit 672f9ff) in a development environment, and then this can be reproduced with both the purelib-and-platlib test case in this repo and with SciPy main:

$ cd tests/packages/purelib-and-platlib
$ python -m build --wheel . --no-isolation --skip-dependency-check
....
[1/1] purelib_and_platlib-1.0.0.data/purelib/pure.pyy                                
Successfully built purelib_and_platlib-1.0.0-py3-none-linux_x86_64.whl

Meson produced this intro-install_plan.json:

{
   "targets":{
      "/home/rgommers/code/tmp/meson-python/tests/packages/purelib-and-platlib/build/plat.cpython-310-x86_64-linux-gnu.so":{
         "destination":"{py_platlib}/plat.cpython-310-x86_64-linux-gnu.so",
         "tag":"runtime"
      }
   },
   "python":{
      "/home/rgommers/code/tmp/meson-python/tests/packages/purelib-and-platlib/pure.py":{
         "destination":"{py_purelib}/pure.py",
         "tag":"runtime"
      }
   }
}

That looks correct. The platlib part of the path gets lost though:

$ cd dist
$ unzip purelib_and_platlib-1.0.0-py3-none-linux_x86_64.whl
$ tree .
.
├── plat.cpython-310-x86_64-linux-gnu.so
├── purelib_and_platlib-1.0.0.data
│   └── purelib
│       └── pure.py
├── purelib_and_platlib-1.0.0.dist-info
│   ├── METADATA
│   ├── RECORD
│   └── WHEEL
└── purelib_and_platlib-1.0.0-py3-none-linux_x86_64.whl

The offending change is gh-130.

Running the test suite yields a failure too, which looks potentially relevant but is failing also with commit 58b0034 (the last one before gh-130):

Details

___________________________________________ test_configure_data ____________________________________________

wheel_configure_data = PosixPath('/tmp/pytest-of-rgommers/pytest-11/test0/mesonpy-test-i_23__v1/configure_data-1.0.0-py3-none-any.whl')

    def test_configure_data(wheel_configure_data):
        artifact = wheel.wheelfile.WheelFile(wheel_configure_data)
    
>       assert wheel_contents(artifact) == {
            'configure_data-1.0.0.data/platlib/configure_data.py',
            'configure_data-1.0.0.dist-info/METADATA',
            'configure_data-1.0.0.dist-info/RECORD',
            'configure_data-1.0.0.dist-info/WHEEL',
        }
E       AssertionError: assert {'configure_d...gure_data.py'} == {'configure_d...t-info/WHEEL'}
E         Extra items in the left set:
E         'configure_data.py'
E         Extra items in the right set:
E         'configure_data-1.0.0.data/platlib/configure_data.py'
E         Use -v to get more diff

tests/test_wheel.py:103: AssertionError
------------------------------------------ Captured stdout setup -------------------------------------------
! Using Meson to generate the project metadata (no `project` section in pyproject.toml)
+ meson setup --native-file=/home/rgommers/code/tmp/meson-python/tests/packages/configure-data/.mesonpy-native-file.ini -Ddebug=false -Doptimization=2 --prefix=/home/rgommers/anaconda3/envs/scipy-dev /home/rgommers/code/tmp/meson-python/tests/packages/configure-data /home/rgommers/code/tmp/meson-python/tests/packages/configure-data/.mesonpy-6wwbyiow/build
The Meson build system
Version: 0.63.99
Source dir: /home/rgommers/code/tmp/meson-python/tests/packages/configure-data
Build dir: /home/rgommers/code/tmp/meson-python/tests/packages/configure-data/.mesonpy-6wwbyiow/build
Build type: native build
Project name: configure-data
Project version: 1.0.0
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program python found: YES (/home/rgommers/anaconda3/envs/scipy-dev/bin/python)
Configuring configure_data.py using configuration
Build targets in project: 0

configure-data 1.0.0

  User defined options
    Native files: /home/rgommers/code/tmp/meson-python/tests/packages/configure-data/.mesonpy-native-file.ini
    debug       : false
    optimization: 2
    prefix      : /home/rgommers/anaconda3/envs/scipy-dev

Found ninja-1.10.2.git.kitware.jobserver-1 at /home/rgommers/anaconda3/envs/scipy-dev/bin/ninja
+ meson compile
ninja: no work to do.
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: /home/rgommers/anaconda3/envs/scipy-dev/bin/ninja
+ meson install --destdir /home/rgommers/code/tmp/meson-python/tests/packages/configure-data/.mesonpy-6wwbyiow/install
ninja: Entering directory `/home/rgommers/code/tmp/meson-python/tests/packages/configure-data/.mesonpy-6wwbyiow/build'
ninja: no work to do.
Installing /home/rgommers/code/tmp/meson-python/tests/packages/configure-data/.mesonpy-6wwbyiow/build/configure_data.py to /home/rgommers/code/tmp/meson-python/tests/packages/configure-data/.mesonpy-6wwbyiow/install/home/rgommers/anaconda3/envs/scipy-dev/lib/python3.10/site-packages
Copying files to wheel...
[0/0] configure_data.py

test_purelib_and_platlib does not fail, because it has an xfail. However, the Meson bug is now fixed (that was the issue with absolute paths being present in intro-install_plan.json I believe). When I uncomment it I get this failure, similar to the configure test failure result above:

Details

_________________________________________ test_purelib_and_platlib _________________________________________

wheel_purelib_and_platlib = PosixPath('/tmp/pytest-of-rgommers/pytest-15/test0/mesonpy-test-ddnnbcp3/purelib_and_platlib-1.0.0-py3-none-any.whl')

    def test_purelib_and_platlib(wheel_purelib_and_platlib):
        artifact = wheel.wheelfile.WheelFile(wheel_purelib_and_platlib)
    
        expecting = {
            f'plat{EXT_SUFFIX}',
            'purelib_and_platlib-1.0.0.data/purelib/pure.py',
            'purelib_and_platlib-1.0.0.dist-info/METADATA',
            'purelib_and_platlib-1.0.0.dist-info/RECORD',
            'purelib_and_platlib-1.0.0.dist-info/WHEEL',
        }
        if platform.system() == 'Windows':
            expecting.add('plat{}'.format(EXT_SUFFIX.replace('pyd', 'dll.a')))
    
>       assert wheel_contents(artifact) == expecting
E       AssertionError: assert {'pure.py', '...t-info/WHEEL'} == {'plat.cpytho...t-info/WHEEL'}
E         Extra items in the left set:
E         'purelib_and_platlib-1.0.0.data/platlib/plat.cpython-310-x86_64-linux-gnu.so'
E         'pure.py'
E         Extra items in the right set:
E         'purelib_and_platlib-1.0.0.data/purelib/pure.py'
E         'plat.cpython-310-x86_64-linux-gnu.so'
E         Use -v to get more diff

The purelib-and-platlib test itself seems wrong, it's unclear why there is no platlib/ in f'plat{EXT_SUFFIX}'.

I'm looking into this.

[rgommers]

The purelib-and-platlib test itself seems wrong, it's unclear why there is no platlib/ in f'plat{EXT_SUFFIX}'.

Ah, what is going on here is that a normal project name and installing into site-packages/<package-name> is missing.

There's only two tests using extension_module:

$ rg extension_module
link-against-local-lib/meson.build
15:py.extension_module(

purelib-and-platlib/meson.build
11:py.extension_module(

and the first one is niche, and the second one not matching what is expected for a normal Python package. We should add a more normal test case.

[FFY00]

Sorry, this should have gotten a test a long time ago, I don't know why it didn't ☹️

[FFY00]

Okay, the issue is that Meson returns the full path, and then we think the project is pure.

[rgommers]

Okay, the issue is that Meson returns the full path, and then we think the project is pure.

It's not, it does happen to work with the full path (which was a meson 0.63.x bug). The destination entries of the intro-install_plan.json in the issue description do not have an absolute path.

I am working on a test case for this, mirroring how SciPy uses features and directory layout. I'll submit that. It's probably easier for you to take it from there.

[FFY00]

Are you sure? I am trying it locally and it does not like Meson returning the absolute path, which my local Meson version does. It messes up is_pure.

https://github.com/FFY00/meson-python/blob/672f9ff3ae0fceb04a0163d26a05a636071b9a7b/mesonpy/__init__.py#L814-L827

[FFY00]

(there may still be another issue)

[FFY00]

Okay, that does not influence the Python and ABI tags, but does the platform tag, which is currently wrong.

[rgommers]

You need to try with Meson master. is_pure is correct, it's False. An issue is that is_pure is not passed to _WheelBuilder at all, so _WheelBuilder.abi_tag happily returns none, and there's no check anywhere that at least concluded that this is impossible. This bug is somewhere in the part of _select_abi_tag that you refactored in gh-130.

(that's how far I got today, more tomorrow)

[FFY00]

I have opened #143 which right now fixes the is_pure detection on absolute paths. I am now looking into the Python and ABI tags.