Skip to content

Fix pyzfs build #17480

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Fix pyzfs build #17480

wants to merge 2 commits into from

Conversation

martin-rueegg
Copy link
Contributor

@martin-rueegg martin-rueegg commented Jun 21, 2025 •
edited
Loading

Motivation and Context

Description

For pyzfs, the evaluation of the library path is done differently at different places:

  • RPM uses primarily distutils.sysconfig and sysconfig as a fallback, while
  • Config uses primarily sysconfig and distutils.sysconfig as a fallback.
  • Furthermore, Config also uses posix_prefix for the scheme parameter of sysconfig.get_path(), while RPM does not.
  • The issue was introduced by 71216b9 and applies to at least the following banches:
    zfs-2.2.x
    zfs-2.3.x
  • Since distutils has been deprecated, the attempt was to update the RPM path evaluation, rather than vice versa.

Open questions

Library Path does not exist on the local system

On all four tested systems (see #16155 (comment)), the path that is evaluated by ax_python_devel.m4 does not exist on the system.

For the build process, it does not matter. As the path is relative to the build directory in /tmp.

However, I don't know why the RPMs are built and if that path is used during installation of pyzfs' RPM. In which case the library might be installed in a place where it is not found?

On Debian and Ubuntu, should the environment-variable DEB_PYTHON_INSTALL_LAYOUT=deb be set? See explanation and implementation fpr further details.

How Has This Been Tested?

Since this change is build-related, the main test is the tests run here on github.

I've also tested it on my Ubuntu 24.04 installation, with python3.12 and distutils installed:

The following output was produced on that build system with the test script provided here.

sysconfig.get_path.purelib                           = /usr/local/lib/python3.12/dist-packages       = ok
sysconfig.get_path.purelib.posix_prefix              = /usr/lib/python3.12/site-packages             = n/a
sysconfig.get_path.purelib.posix_local               = /usr/local/lib/python3.12/dist-packages       = ok
sysconfig.get_path.purelib.auto.base                 = /lib/python3.12/site-packages                 = n/a
sysconfig.get_path.purelib.auto.platbase             = /lib/python3.12/site-packages                 = n/a
sysconfig.get_path.include                           = /usr/include/python3.12                       = ok
distutils.sysconfig.get_python_inc                   = /usr/include/python3.12                       = ok
distutils.sysconfig.get_python_inc(plat_specific=1)  = /usr/include/python3.12                       = ok
distutils.sysconfig.get_python_lib(0,0)              = /usr/lib/python3/dist-packages                = ok
distutils.sysconfig.get_python_lib(1,0)              = /usr/lib/python3/dist-packages                = ok
distutils.sysconfig.get_python_lib()                 = /usr/lib/python3/dist-packages                = ok

Test Script Version: 1.1

hostnamectl:
Operating System: Ubuntu 24.04.2 LTS
          Kernel: Linux 5.15.167.4-microsoft-standard-WSL2+

lsb_release:
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 24.04.2 LTS
Release:        24.04
Codename:       noble

/etc/lsb-release:
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=24.04
DISTRIB_CODENAME=noble
DISTRIB_DESCRIPTION="Ubuntu 24.04.2 LTS"

/etc/os-release:
PRETTY_NAME="Ubuntu 24.04.2 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04.2 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo

APT packages:
python3-distro
python3-distro-info

PIP packages:
distlib==0.3.9
distro==1.9.0
distro-info==1.7+build1

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Performance enhancement (non-breaking change which improves efficiency)
  • Code cleanup (non-breaking change which makes code smaller or more readable)
  • Quality assurance (non-breaking change which makes the code more robust against bugs)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
  • Documentation (a change to man pages or other documentation)

Checklist:

@martin-rueegg martin-rueegg mentioned this pull request Jun 21, 2025
Open
@martin-rueegg
pyzfs: Update ax_python_devel.m4 to serial 37
0b93957 
Fixes an obvious type, where a variable was missing the required
leading dollar sign ($)

Signed-off-by: Martin Rüegg <martin.rueegg@metaworx.ch>
@martin-rueegg
Copy link
Contributor Author

Build fails on Fedora 41:

Build fails on AlmaLinux 10:

@martin-rueegg
pyzfs: Adapt python lib directory evaluation from ax_python_devel.m4
e772f5b 
71216b9 introduced a regression
on debian/ubuntu systems during build.

The reason being, that building the RPM for pyzfs was using
a different library path than building the library itself.
This is now harmonized.

Closes: openzfs#16155

Signed-off-by: Martin Rüegg <martin.rueegg@metaworx.ch>
@martin-rueegg
Copy link
Contributor Author

Fedora 41

  • pyzfs
    • /tmp/zfs-build-zfs-NR92g5cO/BUILD/zfs-2.3.99-build/BUILDROOT
    • /usr/lib/python3.13/site-packages/libzfs_core
  • RPM
    • /tmp/zfs-build-zfs-NR92g5cO/BUILD/zfs-2.3.99-build/BUILDROOT/lib64/python3.13/site-packages/libzfs_core/
    • /lib64/python3.13/site-packages/libzfs_core/

It follows, that

  • the --prefix parameter has not been considered
    => Use %{_prefix} (for rpm spec) rather than $prefix}
  • the platform specific directory was used for rpm (PYTHON_PLATFORM_SITE_PKG instead of PYTHON_SITE_PKG)
    => Use purelib rather than platlib

@martin-rueegg
Copy link
Contributor Author

Ok, build was successful on all tested platforms.

@martin-rueegg martin-rueegg marked this pull request as ready for review June 22, 2025 21:01
@martin-rueegg
Copy link
Contributor Author

martin-rueegg commented Jun 22, 2025 •
edited
Loading

@behlendorf could you take a look at this, since you authored #16129?

@amotin amotin added the Status: Code Review Needed Ready for review and testing label Jun 23, 2025
@amotin amotin linked an issue Jun 24, 2025 that may be closed by this pull request
OpenZFS 2.2.4 build fail on Debian 12 #16155
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
Status: Code Review Needed Ready for review and testing
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

OpenZFS 2.2.4 build fail on Debian 12
2 participants
@martin-rueegg @amotin