Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Several tests related to Python distributions fail on Fedora Linux on non-x86_64 architectures #1145

Closed
musicinmybrain opened this issue Dec 14, 2023 · 9 comments · Fixed by #1175

Comments

@musicinmybrain
Copy link
Contributor

I’m working on updating the hatch and python-hatchling packages in Fedora Rawhide to the latest versions (1.8.1 and 1.20.0, respectively). On architectures other than x86_64, several tests related to the new Python distribution support fail. Here’s an example from aarch64:

=================================== FAILURES ===================================
_________________________ TestGetInstalled.test_order __________________________

self = <tests.python.test_core.TestGetInstalled object at 0xffff861921e0>
temp_dir = Path('/tmp/tmpb_72kh9s')

    def test_order(self, temp_dir):
        manager = PythonManager(temp_dir)
    
        for name in DISTRIBUTIONS:
>           dist = get_distribution(name)

/builddir/build/BUILD/hatch-hatch-v1.8.1/tests/python/test_core.py:71: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

name = '3.7', source = '', variant = ''

    def get_distribution(name: str, source: str = '', variant: str = '') -> Distribution:
        if source:
            return _get_distribution_class(source)(name, source)
    
        if name not in DISTRIBUTIONS:
            message = f'Unknown distribution: {name}'
            raise PythonDistributionUnknownError(message)
    
        arch = platform.machine().lower()
        if sys.platform == 'win32':
            system = 'windows'
            abi = 'msvc'
        elif sys.platform == 'darwin':
            system = 'macos'
            abi = ''
        else:
            system = 'linux'
            abi = 'gnu' if any(platform.libc_ver()) else 'musl'
    
        if not variant:
            variant = _get_default_variant(name, system, arch, abi)
    
        key = (system, arch, abi, variant)
    
        keys: dict[tuple, str] = DISTRIBUTIONS[name]
        if key not in keys:
            message = f'Could not find a default source for {name=} {system=} {arch=} {abi=} {variant=}'
>           raise PythonDistributionResolutionError(message)
E           hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.7' system='linux' arch='aarch64' abi='gnu' variant=''

/builddir/build/BUILDROOT/hatch-1.8.1-1.fc40.aarch64/usr/lib/python3.12/site-packages/hatch/python/resolve.py:153: PythonDistributionResolutionError
___________________________ test_variants[linux-v4] ____________________________

platform = <hatch.utils.platform.Platform object at 0xffff87e470e0>
system = 'linux', variant = 'v4'

    @pytest.mark.parametrize(
        ('system', 'variant'),
        [
            ('windows', 'shared'),
            ('windows', 'static'),
            ('linux', 'v1'),
            ('linux', 'v2'),
            ('linux', 'v3'),
            ('linux', 'v4'),
        ],
    )
    def test_variants(platform, system, variant):
        if platform.name != system:
            pytest.skip(f'Skipping test for: {system}')
    
        with EnvVars({f'HATCH_PYTHON_VARIANT_{system.upper()}': variant}):
>           dist = get_distribution('3.11')

/builddir/build/BUILD/hatch-hatch-v1.8.1/tests/python/test_resolve.py:55: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

name = '3.11', source = '', variant = 'v4'

    def get_distribution(name: str, source: str = '', variant: str = '') -> Distribution:
        if source:
            return _get_distribution_class(source)(name, source)
    
        if name not in DISTRIBUTIONS:
            message = f'Unknown distribution: {name}'
            raise PythonDistributionUnknownError(message)
    
        arch = platform.machine().lower()
        if sys.platform == 'win32':
            system = 'windows'
            abi = 'msvc'
        elif sys.platform == 'darwin':
            system = 'macos'
            abi = ''
        else:
            system = 'linux'
            abi = 'gnu' if any(platform.libc_ver()) else 'musl'
    
        if not variant:
            variant = _get_default_variant(name, system, arch, abi)
    
        key = (system, arch, abi, variant)
    
        keys: dict[tuple, str] = DISTRIBUTIONS[name]
        if key not in keys:
            message = f'Could not find a default source for {name=} {system=} {arch=} {abi=} {variant=}'
>           raise PythonDistributionResolutionError(message)
E           hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='aarch64' abi='gnu' variant='v4'

/builddir/build/BUILDROOT/hatch-1.8.1-1.fc40.aarch64/usr/lib/python3.12/site-packages/hatch/python/resolve.py:153: PythonDistributionResolutionError
___________________________ test_variants[linux-v3] ____________________________

platform = <hatch.utils.platform.Platform object at 0xffff87e470e0>
system = 'linux', variant = 'v3'

    @pytest.mark.parametrize(
        ('system', 'variant'),
        [
            ('windows', 'shared'),
            ('windows', 'static'),
            ('linux', 'v1'),
            ('linux', 'v2'),
            ('linux', 'v3'),
            ('linux', 'v4'),
        ],
    )
    def test_variants(platform, system, variant):
        if platform.name != system:
            pytest.skip(f'Skipping test for: {system}')
    
        with EnvVars({f'HATCH_PYTHON_VARIANT_{system.upper()}': variant}):
>           dist = get_distribution('3.11')

/builddir/build/BUILD/hatch-hatch-v1.8.1/tests/python/test_resolve.py:55: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

name = '3.11', source = '', variant = 'v3'

    def get_distribution(name: str, source: str = '', variant: str = '') -> Distribution:
        if source:
            return _get_distribution_class(source)(name, source)
    
        if name not in DISTRIBUTIONS:
            message = f'Unknown distribution: {name}'
            raise PythonDistributionUnknownError(message)
    
        arch = platform.machine().lower()
        if sys.platform == 'win32':
            system = 'windows'
            abi = 'msvc'
        elif sys.platform == 'darwin':
            system = 'macos'
            abi = ''
        else:
            system = 'linux'
            abi = 'gnu' if any(platform.libc_ver()) else 'musl'
    
        if not variant:
            variant = _get_default_variant(name, system, arch, abi)
    
        key = (system, arch, abi, variant)
    
        keys: dict[tuple, str] = DISTRIBUTIONS[name]
        if key not in keys:
            message = f'Could not find a default source for {name=} {system=} {arch=} {abi=} {variant=}'
>           raise PythonDistributionResolutionError(message)
E           hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='aarch64' abi='gnu' variant='v3'

/builddir/build/BUILDROOT/hatch-1.8.1-1.fc40.aarch64/usr/lib/python3.12/site-packages/hatch/python/resolve.py:153: PythonDistributionResolutionError
___________________________ test_variants[linux-v2] ____________________________

platform = <hatch.utils.platform.Platform object at 0xffff87e470e0>
system = 'linux', variant = 'v2'

    @pytest.mark.parametrize(
        ('system', 'variant'),
        [
            ('windows', 'shared'),
            ('windows', 'static'),
            ('linux', 'v1'),
            ('linux', 'v2'),
            ('linux', 'v3'),
            ('linux', 'v4'),
        ],
    )
    def test_variants(platform, system, variant):
        if platform.name != system:
            pytest.skip(f'Skipping test for: {system}')
    
        with EnvVars({f'HATCH_PYTHON_VARIANT_{system.upper()}': variant}):
>           dist = get_distribution('3.11')

/builddir/build/BUILD/hatch-hatch-v1.8.1/tests/python/test_resolve.py:55: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

name = '3.11', source = '', variant = 'v2'

    def get_distribution(name: str, source: str = '', variant: str = '') -> Distribution:
        if source:
            return _get_distribution_class(source)(name, source)
    
        if name not in DISTRIBUTIONS:
            message = f'Unknown distribution: {name}'
            raise PythonDistributionUnknownError(message)
    
        arch = platform.machine().lower()
        if sys.platform == 'win32':
            system = 'windows'
            abi = 'msvc'
        elif sys.platform == 'darwin':
            system = 'macos'
            abi = ''
        else:
            system = 'linux'
            abi = 'gnu' if any(platform.libc_ver()) else 'musl'
    
        if not variant:
            variant = _get_default_variant(name, system, arch, abi)
    
        key = (system, arch, abi, variant)
    
        keys: dict[tuple, str] = DISTRIBUTIONS[name]
        if key not in keys:
            message = f'Could not find a default source for {name=} {system=} {arch=} {abi=} {variant=}'
>           raise PythonDistributionResolutionError(message)
E           hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='aarch64' abi='gnu' variant='v2'

/builddir/build/BUILDROOT/hatch-1.8.1-1.fc40.aarch64/usr/lib/python3.12/site-packages/hatch/python/resolve.py:153: PythonDistributionResolutionError
___________________________ test_variants[linux-v1] ____________________________

platform = <hatch.utils.platform.Platform object at 0xffff87e470e0>
system = 'linux', variant = 'v1'

    @pytest.mark.parametrize(
        ('system', 'variant'),
        [
            ('windows', 'shared'),
            ('windows', 'static'),
            ('linux', 'v1'),
            ('linux', 'v2'),
            ('linux', 'v3'),
            ('linux', 'v4'),
        ],
    )
    def test_variants(platform, system, variant):
        if platform.name != system:
            pytest.skip(f'Skipping test for: {system}')
    
        with EnvVars({f'HATCH_PYTHON_VARIANT_{system.upper()}': variant}):
>           dist = get_distribution('3.11')

/builddir/build/BUILD/hatch-hatch-v1.8.1/tests/python/test_resolve.py:55: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

name = '3.11', source = '', variant = 'v1'

    def get_distribution(name: str, source: str = '', variant: str = '') -> Distribution:
        if source:
            return _get_distribution_class(source)(name, source)
    
        if name not in DISTRIBUTIONS:
            message = f'Unknown distribution: {name}'
            raise PythonDistributionUnknownError(message)
    
        arch = platform.machine().lower()
        if sys.platform == 'win32':
            system = 'windows'
            abi = 'msvc'
        elif sys.platform == 'darwin':
            system = 'macos'
            abi = ''
        else:
            system = 'linux'
            abi = 'gnu' if any(platform.libc_ver()) else 'musl'
    
        if not variant:
            variant = _get_default_variant(name, system, arch, abi)
    
        key = (system, arch, abi, variant)
    
        keys: dict[tuple, str] = DISTRIBUTIONS[name]
        if key not in keys:
            message = f'Could not find a default source for {name=} {system=} {arch=} {abi=} {variant=}'
>           raise PythonDistributionResolutionError(message)
E           hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='aarch64' abi='gnu' variant='v1'

/builddir/build/BUILDROOT/hatch-1.8.1-1.fc40.aarch64/usr/lib/python3.12/site-packages/hatch/python/resolve.py:153: PythonDistributionResolutionError
=============================== warnings summary ===============================
tests/backend/builders/test_sdist.py::TestBuildStandard::test_include_readme
tests/backend/builders/test_sdist.py::TestBuildStandard::test_project_file_always_included
tests/backend/builders/test_sdist.py::TestBuildStandard::test_include_license_files
tests/backend/builders/test_sdist.py::TestBuildStandard::test_no_strict_naming
tests/backend/builders/test_sdist.py::TestBuildStandard::test_readme_always_included
tests/backend/builders/test_sdist.py::TestBuildStandard::test_config_file_always_included
tests/backend/builders/test_sdist.py::TestBuildStandard::test_license_files_always_included
tests/backend/builders/test_sdist.py::TestBuildStandard::test_include_project_file
  /usr/lib64/python3.12/tarfile.py:2220: DeprecationWarning: Python 3.14 will, by default, filter extracted tar archives and reject files or modify their metadata. Use the filter argument to control this behavior.
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests/python/test_core.py::TestGetInstalled::test_order - hatch.errors...
FAILED tests/python/test_resolve.py::test_variants[linux-v4] - hatch.errors.P...
FAILED tests/python/test_resolve.py::test_variants[linux-v3] - hatch.errors.P...
FAILED tests/python/test_resolve.py::test_variants[linux-v2] - hatch.errors.P...
FAILED tests/python/test_resolve.py::test_variants[linux-v1] - hatch.errors.P...
=== 5 failed, 1617 passed, 98 skipped, 176 deselected, 8 warnings in 37.98s ====

Full build logs

builder-live.aarch64.txt
builder-live.i386.txt
builder-live.ppc64le.txt
builder-live.s390x.txt


It seems like one problem may be that test_variants is testing for x86_64 feature levels, but using the host architecture. I haven’t investigated what’s going on with TestGetInstalled::test_order.

For now, I’m just skipping these tests on non-x86_64 platforms.

@ofek
Copy link
Sponsor Collaborator

ofek commented Dec 14, 2023

Thanks for bringing this to my attention! I don't quite know how to debug this

@ofek
Copy link
Sponsor Collaborator

ofek commented Dec 18, 2023

Would you mind pasting the output of those platform inspection standard library functions? I might just be looking for the wrong stuff.

@musicinmybrain
Copy link
Contributor Author

Would you mind pasting the output of those platform inspection standard library functions? I might just be looking for the wrong stuff.

Is this what you’re looking for? (Fedora Rawhide, all primary architectures):

x86_64

+ /usr/bin/python3 -c 'import platform; print(repr(platform.machine()))'
'x86_64'
+ /usr/bin/python3 -c 'import platform; print(repr(platform.libc_ver()))'
('glibc', '2.38.9000')

i686

'i686'
('glibc', '2.38.9000')

ppc64le

'ppc64le'
('glibc', '2.38.9000')

s390x

's390x'
('glibc', '2.38.9000')

aarch64

'aarch64'
('glibc', '2.38.9000')

@bnavigator
Copy link

openSUSE:

# LegacyX86

hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='i686' abi='gnu' variant='v4'

# ARM
hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='aarch64' abi='gnu' variant='v4'
hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='armv7l' abi='gnu' variant='v4'
hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='armv6l' abi='gnu' variant='v4'

# PowerPC
hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='ppc64le' abi='gnu' variant='v4'

# RISCV
hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='riscv64' abi='gnu' variant='v4'

# zSystems
hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='s390x' abi='gnu' variant='v4'

@ofek
Copy link
Sponsor Collaborator

ofek commented Dec 25, 2023

Please let me know if this is now fixed

edit: specifically @bnavigator

hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='i686' abi='gnu' variant='v4'

There are no 32-bit distributions that are available so this will not be fixed. Even when there are official compiled distributions I'm not sure PSF would invest in maintenance of that architecture either.

hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='aarch64' abi='gnu' variant='v4'
hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='armv7l' abi='gnu' variant='v4'
hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='armv6l' abi='gnu' variant='v4'

This I didn't fix actually I just realized. Can you please explain the situation in which platform.machine() would not return aarch64?

@bnavigator
Copy link

There are still thousands of Raspberry Pis and similar barebones with 32-bit ARM out there. The official recommended Raspberry Pi OS ist still 32-bit:
https://www.raspberrypi.com/software/operating-systems/#raspberry-pi-os-32-bit

Also check

There was a survey for openSUSE about ARMv7 use cases in 2020(!):
https://en.opensuse.org/Portal:15.3/Surveys/ARM_Usecases

@bnavigator
Copy link

bnavigator commented Dec 25, 2023

On Intel 32bit (ix86), aarch64, ppc64le, and s390x, the unit test error now shifted to:

[   64s] _______________________ TestErrors.test_resolution_error _______________________
[   64s] 
[   64s] self = <tests.python.test_resolve.TestErrors object at 0xf57404a8>
[   64s] platform = <hatch.utils.platform.Platform object at 0xf676ae80>
[   64s] 
[   64s]     @pytest.mark.skipif(sys.platform == 'darwin', reason='No variants for macOS')
[   64s]     def test_resolution_error(self, platform):
[   64s]         with EnvVars({f'HATCH_PYTHON_VARIANT_{platform.name.upper()}': 'foo'}), pytest.raises(
[   64s]             PythonDistributionResolutionError,
[   64s]             match=f"Could not find a default source for name='3.11' system='{platform.name}' arch=",
[   64s]         ):
[   64s] >           get_distribution('3.11')
[   64s] E           Failed: DID NOT RAISE <class 'hatch.errors.PythonDistributionResolutionError'>
[   64s] 
[   64s] /home/abuild/rpmbuild/BUILD/hatch-hatch-v1.9.1/tests/python/test_resolve.py:21: Failed
[   64s] =========================== short test summary info ============================
[   64s] FAILED tests/python/test_resolve.py::TestErrors::test_resolution_error - Fail...
[   64s] ========== 1 failed, 1799 passed, 107 skipped, 3 deselected in 46.85s ==========

armv7 still fails in the mentioned and additional tests, all related to the distributions choice:

[   64s] ________________________ ERROR at setup of test_binary _________________________
[   64s] 
[   64s] compatible_python_distributions = ()
[   64s] 
[   64s]     @pytest.fixture
[   64s]     def dist_name(compatible_python_distributions):
[   64s] >       return secrets.choice(compatible_python_distributions)
[   64s] 
[   64s] /home/abuild/rpmbuild/BUILD/hatch-hatch-v1.9.1/tests/cli/python/conftest.py:30: 
[   64s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   64s] 
[   64s] self = <random.SystemRandom object at 0x1d720b0>, seq = ()
[   64s] 
[   64s]     def choice(self, seq):
[   64s]         """Choose a random element from a non-empty sequence."""
[   64s]         # raises IndexError if seq is empty
[   64s] >       return seq[self._randbelow(len(seq))]
[   64s] E       IndexError: tuple index out of range
[   64s] 
[   64s] /usr/lib/python3.9/random.py:346: IndexError
...
[   65s] FAILED tests/cli/python/test_remove.py::test_basic - hatch.errors.PythonDistr...
[   65s] FAILED tests/cli/python/test_remove.py::test_all - hatch.errors.PythonDistrib...
[   65s] FAILED tests/cli/python/test_show.py::test_nothing_installed - AssertionError...
[   65s] FAILED tests/cli/python/test_show.py::test_all_installed - AssertionError: as...
[   65s] FAILED tests/cli/python/test_update.py::test_all - hatch.errors.PythonDistrib...
[   65s] FAILED tests/python/test_core.py::TestGetInstalled::test_not_a_directory - ha...
[   65s] FAILED tests/python/test_core.py::TestGetInstalled::test_no_metadata_file - h...
[   65s] FAILED tests/python/test_core.py::TestGetInstalled::test_no_python_path - hat...
[   65s] FAILED tests/python/test_resolve.py::test_variants[linux-v1] - hatch.errors.P...
[   65s] FAILED tests/python/test_resolve.py::test_variants[linux-v2] - hatch.errors.P...
[   65s] FAILED tests/python/test_resolve.py::test_variants[linux-v3] - hatch.errors.P...
[   65s] FAILED tests/python/test_resolve.py::test_variants[linux-v4] - hatch.errors.P...
[   65s] ERROR tests/cli/python/test_find.py::test_binary - IndexError: tuple index ou...
[   65s] ERROR tests/cli/python/test_find.py::test_parent - IndexError: tuple index ou...
[   65s] ERROR tests/cli/python/test_install.py::test_incompatible_single - IndexError...
[   65s] ERROR tests/cli/python/test_install.py::test_already_installed_latest - Index...
[   65s] ERROR tests/cli/python/test_install.py::test_already_installed_update_disabled
[   65s] ERROR tests/cli/python/test_install.py::test_already_in_path[in_current_path]
[   65s] ERROR tests/cli/python/test_install.py::test_already_in_path[in_new_path] - I...
[   65s] ERROR tests/cli/python/test_install.py::test_private - IndexError: tuple inde...
[   65s] ERROR tests/cli/python/test_install.py::test_specific_location - IndexError: ...
[   65s] ERROR tests/cli/python/test_remove.py::test_specific_location - IndexError: t...
[   65s] ERROR tests/cli/python/test_show.py::test_some_installed - IndexError: tuple ...
[   65s] ERROR tests/cli/python/test_show.py::test_specific_location - IndexError: tup...
[   65s] ERROR tests/cli/python/test_show.py::test_outdated - IndexError: tuple index ...
[   65s] ERROR tests/cli/python/test_update.py::test_basic - IndexError: tuple index o...
[   65s] ERROR tests/cli/python/test_update.py::test_specific_location - IndexError: t...

If you have an openSUSE buildservice account, you can see the build and test suite outputs here: https://build.opensuse.org/package/show/home:bnavigator:branches:hatchdebug/python-hatch

@ofek
Copy link
Sponsor Collaborator

ofek commented Dec 25, 2023

I fixed the one test that was failing #1177

Unfortunately I have no way to fix the ARM 32-bit tests because there is no available distribution https://github.com/indygreg/python-build-standalone/releases

@musicinmybrain
Copy link
Contributor Author

On Intel 32bit (ix86), aarch64, ppc64le, and s390x, the unit test error now shifted to:

[   64s] _______________________ TestErrors.test_resolution_error _______________________

Fedora no longer supports 32-bit ARM, so I can’t comment on that, but on 32-bit x86 (i686) I do see the same thing @bnavigator reported on 1.9.1, and I can confirm it’s fixed by #1177. With that patch, the tests pass on all of our primary architectures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants