Figure out how to incorporate Fedora's (and openSUSE's) lib64 patches on Python 3.7/3.8 #110

hroncok · 2022-01-21T22:50:26Z

See pylint-dev/pylint#5704 and pypa/setuptools#2896 (comment)

Apparently, I have completely forgotten that on Python < 3.9, we do patch distutils in one more way: We replace /usr/lib with /usr/lib64 on 64bit architectures.

See our patches:

Python 3.8: fedora-python/cpython@31397a8
Python 3.7: fedora-python/cpython@76a09bc

I am opening this issue to track this problem. I don't expect you to fix it, I plan to have a look, but not today.

The text was updated successfully, but these errors were encountered:

hroncok · 2022-01-24T12:47:50Z

Looking at the current implementation of pypa/distutils, it seems we only need to set sys.platlibdir to "lib64" and it should work as expected :)

… /usr/lib/ Only used when "%{_lib}" == "lib64". Also sets sys.platlibdir to "lib64" for pypa/distutils#110 Co-authored-by: David Malcolm <dmalcolm@redhat.com> Co-authored-by: Thomas Spura <tomspur@fedoraproject.org> Co-authored-by: Slavek Kabrda <bkabrda@redhat.com> Co-authored-by: Matej Stuchlik <mstuchli@redhat.com> Co-authored-by: Tomas Orsava <torsava@redhat.com> Co-authored-by: Charalampos Stratakis <cstratak@redhat.com> Co-authored-by: Petr Viktorin <pviktori@redhat.com> Co-authored-by: Miro Hrončok <miro@hroncok.cz> Co-authored-by: Iryna Shcherbina <shcherbina.iryna@gmail.com>

hroncok · 2022-01-24T13:04:02Z

Work in progress PR for python3.8: https://src.fedoraproject.org/rpms/python3.8/pull-request/54

jaraco · 2022-01-24T13:23:35Z

I'll confess, I'm slightly uneasy with altering a value in sys, but when I look at the schemes, {platlibdir} resolves directly from sys.platlibdir, so I agree with the conclusion. I wonder what effect changing that value will have on other environments, such as venv or virtualenv.

hroncok · 2022-01-24T13:25:41Z

Yeah, my idea is to set it, and try to see what it breaks.

… /usr/lib/ Only used when "%{_lib}" == "lib64". Also sets sys.platlibdir to "lib64" for pypa/distutils#110 Co-authored-by: David Malcolm <dmalcolm@redhat.com> Co-authored-by: Thomas Spura <tomspur@fedoraproject.org> Co-authored-by: Slavek Kabrda <bkabrda@redhat.com> Co-authored-by: Matej Stuchlik <mstuchli@redhat.com> Co-authored-by: Tomas Orsava <torsava@redhat.com> Co-authored-by: Charalampos Stratakis <cstratak@redhat.com> Co-authored-by: Petr Viktorin <pviktori@redhat.com> Co-authored-by: Miro Hrončok <miro@hroncok.cz> Co-authored-by: Iryna Shcherbina <shcherbina.iryna@gmail.com>

hroncok · 2022-02-17T19:23:49Z

Nothing broke for me, @encukou or @jnsnow. Let's share this with the crowds. Fedora updates:

The updates will soon be installable from the updates-testing repo.

Setuptools v60 and later include a bundled version of distutils, a deprecated standard library scheduled for removal in future versions of Python. Setuptools v60 is only possible to install for Python 3.7 and later. Python has a distutils.sysconfig.get_python_lib() function that returns '/usr/lib/pythonX.Y' on posix systems. RPM-based systems actually use '/usr/lib64/pythonX.Y' instead, so Fedora patches stdlib distutils for Python 3.7 and Python 3.8 to return the correct value. Python 3.9 and later introduce a sys.platlibdir property, which returns the correct value on RPM-based systems. The change to a distutils package not provided by Fedora on Python 3.7 and 3.8 causes a regression in distutils.sysconfig.get_python_lib() that ultimately causes false positives to be emitted by pylint, because it can no longer find the system source libraries. Many Python tools are fairly aggressive about updating setuptools packages, and so even though this package is a fair bit newer than Python 3.7/3.8, it's not entirely unreasonable for a given user to have such a modern package with a fairly old Python interpreter. Updates to Python 3.7 and Python 3.8 are being produced for Fedora which will fix the problem on up-to-date systems. Until then, we can force the loading of platform-provided distutils when running the pylint test. This is the least-invasive yet most comprehensive fix. References: pypa/setuptools#2896 pylint-dev/pylint#5704 pypa/distutils#110 Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20220204221804.2047468-2-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>

jaraco · 2022-02-26T20:05:35Z

That's great. Does that mean this issue is done? Optimistically closing, but feel free to report back if not.

hroncok · 2022-02-26T20:20:46Z

Yes, it is hopefully resolved for Fedora at least.

Setuptools v60 and later include a bundled version of distutils, a deprecated standard library scheduled for removal in future versions of Python. Setuptools v60 is only possible to install for Python 3.7 and later. Python has a distutils.sysconfig.get_python_lib() function that returns '/usr/lib/pythonX.Y' on posix systems. RPM-based systems actually use '/usr/lib64/pythonX.Y' instead, so Fedora patches stdlib distutils for Python 3.7 and Python 3.8 to return the correct value. Python 3.9 and later introduce a sys.platlibdir property, which returns the correct value on RPM-based systems. The change to a distutils package not provided by Fedora on Python 3.7 and 3.8 causes a regression in distutils.sysconfig.get_python_lib() that ultimately causes false positives to be emitted by pylint, because it can no longer find the system source libraries. Many Python tools are fairly aggressive about updating setuptools packages, and so even though this package is a fair bit newer than Python 3.7/3.8, it's not entirely unreasonable for a given user to have such a modern package with a fairly old Python interpreter. Updates to Python 3.7 and Python 3.8 are being produced for Fedora which will fix the problem on up-to-date systems. Until then, we can force the loading of platform-provided distutils when running the pylint test. This is the least-invasive yet most comprehensive fix. References: pypa/setuptools#2896 pylint-dev/pylint#5704 pypa/distutils#110 Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20220204221804.2047468-2-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>

… /usr/lib/ Only used when "%{_lib}" == "lib64". Also sets sys.platlibdir to "lib64" for pypa/distutils#110 Co-authored-by: David Malcolm <dmalcolm@redhat.com> Co-authored-by: Thomas Spura <tomspur@fedoraproject.org> Co-authored-by: Slavek Kabrda <bkabrda@redhat.com> Co-authored-by: Matej Stuchlik <mstuchli@redhat.com> Co-authored-by: Tomas Orsava <torsava@redhat.com> Co-authored-by: Charalampos Stratakis <cstratak@redhat.com> Co-authored-by: Petr Viktorin <pviktori@redhat.com> Co-authored-by: Miro Hrončok <miro@hroncok.cz> Co-authored-by: Iryna Shcherbina <shcherbina.iryna@gmail.com>

This was referenced Jan 21, 2022

false positive 'c-extension-no-member' when running in environment with setuptools>=60.0 pylint-dev/pylint#5704

Closed

Restore local distutils as the default. pypa/setuptools#2896

Merged

jaraco added this to the Support platform-specific behaviors milestone Jan 22, 2022

jaraco closed this as completed Feb 26, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Figure out how to incorporate Fedora's (and openSUSE's) lib64 patches on Python 3.7/3.8 #110

Figure out how to incorporate Fedora's (and openSUSE's) lib64 patches on Python 3.7/3.8 #110

hroncok commented Jan 21, 2022 •

edited

hroncok commented Jan 24, 2022

hroncok commented Jan 24, 2022

jaraco commented Jan 24, 2022

hroncok commented Jan 24, 2022

hroncok commented Feb 17, 2022

jaraco commented Feb 26, 2022

hroncok commented Feb 26, 2022

Figure out how to incorporate Fedora's (and openSUSE's) lib64 patches on Python 3.7/3.8 #110

Figure out how to incorporate Fedora's (and openSUSE's) lib64 patches on Python 3.7/3.8 #110

Comments

hroncok commented Jan 21, 2022 • edited

hroncok commented Jan 24, 2022

hroncok commented Jan 24, 2022

jaraco commented Jan 24, 2022

hroncok commented Jan 24, 2022

hroncok commented Feb 17, 2022

jaraco commented Feb 26, 2022

hroncok commented Feb 26, 2022

hroncok commented Jan 21, 2022 •

edited