Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python distutils is being removed #392

Closed
linuxdude42 opened this issue Sep 26, 2021 · 12 comments · Fixed by #400
Closed

Python distutils is being removed #392

linuxdude42 opened this issue Sep 26, 2021 · 12 comments · Fixed by #400
Assignees
Labels
component:bindings:python Python Bindings task:developer Developer Task
Milestone

Comments

@linuxdude42
Copy link
Contributor

Compiles on fedora rawhide have started throwing the following warning:

bindings/python/setup.py:4: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives from distutils.core import setup

This code needs to be updated to use setuptools.

Platform: Fedora Rawhide, x86_64
MythTV version: master
Package version: if using a packaged version of MythTV, please provide repository/version
Component: Python bindings

What steps will reproduce the bug?

  1. configure
  2. make

How often does it reproduce? Is there a required condition?

Happens every time.

@linuxdude42 linuxdude42 added task:developer Developer Task component:bindings:python Python Bindings labels Sep 26, 2021
@linuxdude42 linuxdude42 added this to the 32.0 milestone Sep 26, 2021
rcrdnalor added a commit to rcrdnalor/mythtv that referenced this issue Oct 10, 2021
for package creation.

In Python 3.10 and 3.11, distutils will be formally marked as deprecated.
See https://www.python.org/dev/peps/pep-0632/
Albeit setuptools uses distutils internally, this will continue to work
in the future, because setuptools includes its own version of distutils.
See https://setuptools.pypa.io/en/latest/deprecated/index.html
Tested on python 3.6+.

Note: Each distribution patches heavily setuptools and distutils, the
package maintainers should check correctness of the new builds.

Although python PEP 518 proposes a new format to specify the build system
for python packages (pyproject.toml), this is not fully implemented in
python3.6 and can't be used.

Fixes MythTV#392
@rcrdnalor rcrdnalor self-assigned this Oct 10, 2021
@rcrdnalor
Copy link
Contributor

Please review pull request #400
I still don't know how to check if Fedora and other OS's then Ubuntu package libmythpython correctly.

@rcrdnalor
Copy link
Contributor

Note. This commit adds an additionally build dependency: python3-setuptools, not committed by now.

@garybuhrmaster
Copy link
Contributor

Please review pull request #400 I still don't know how to check if Fedora and other OS's then Ubuntu package libmythpython correctly.

For Fedora and EL, the requirement for MySQLdb > 1.3 (auto-generated by the build dependency generator) cannot be satisfied with my previous package build scripts (the python3-mysql package on the Fedora/EL variants provides mysqlclient, not mysqldb (as I recall, there was a divergence in the available connectors for python3)). Note that I believe this is a packaging build issue only, and would (likely) not impact manual compile/install, as the MythTV ansible scripts install the appropriate python library for the appropriate system (python3-mysqldb for deb based systems, python3-mysql for dnf based systems).

Perhaps the install_requires could be distro specific, although that ends up being another thing to maintain for OS variants (even if PEP 508 tries to make it a little easier), and is clearly not desirable.

That said, it is easy enough to exclude the auto-generated dependencies when using rpm packaging, so the patch seems to build (I have not tested functionality, but the correct files seem to get installed in the correct places) on Fedora and EL.

@garybuhrmaster
Copy link
Contributor

It occurs to me that configure already checks for (a viable) python-MySQLdb and a python-lxml, so, perhaps, one does not need to include those in the install_requires section (which simplifies other OS library packaging due to different implementations).

@rcrdnalor
Copy link
Contributor

@garybuhrmaster
Thank you for reviewing this pull request.
The 'install_requires' section in the file 'setup.cfg' is primary meant for the 'pypi' archive.
Each OS distribution overrides this with the actual dependencies or add own dependencies..
You are right, the 'MySQLdb' entry is now for python3 'mysqlclient'.

I will either add a complete set of dependencies, i.e.: python pypi packages for
python3-lxml, python3-mysqldb, python3-requests, python3-requests-cache,
python3-simplejson, python3-future, python3-systemd
or completely leave out this section.

@garybuhrmaster
Copy link
Contributor

(another) btw, since python3-setuptools is likely going to be needed going forward (no matter the details of how you deal with install_requires), committing a change now to update .github/workflows/buildmaster.yml now will (likely) let the CI pull-request checks pass going forward as you decide to update the pull request.

@rcrdnalor
Copy link
Contributor

@linuxdude42
Could you please add python3-setuptools to .github/workflows/buildmaster.yml ?
Log file of the github build shows:

Traceback (most recent call last):
  File "setup.py", line 4, in <module>
    import setuptools
ModuleNotFoundError: No module named 'setuptools'
make[1]: *** [python_build] Error 1
Makefile:22: recipe for target 'python_build' failed

At least, the item Install core dependencies (linux) needs this entry.
I have no right to do that. I think I forgot to raise my hand when I was asked for getting the write-rights.

Thank you,
Roland

@garybuhrmaster
Copy link
Contributor

btw, since this is for python3-setuptools, does this mean that the project has officially abandoned python2 support for master (and beyond)? I do not object (in fact, would recommend it), but I do not recall that being officially announced. If the project is now python3 only that may allow some elimination of some hoop jumping in certain constructs, and it can also mean removing (or replacement) for some of the python2 only scripts that were identified in #386.

@rcrdnalor
Copy link
Contributor

Excerpt from https://www.mythtv.org/wiki/Release_Notes_-_31 :

MythTV's python2 support is now considered deprecated and will be removed in a future version

@garybuhrmaster
Copy link
Contributor

The release notes simply stated a "future version". Is v32 that version, or will there be a need for the replacement code of distutils to be able to support python2-setuptools (or whatever name the various distro uses for the python2 setuptools)?

@linuxdude42
Copy link
Contributor Author

V32 will only support python3.

rcrdnalor added a commit to rcrdnalor/mythtv that referenced this issue Oct 24, 2021
from 'setup.cfg' because it is not handled correctly by various
package managers, like Ubuntu. Only the last item is displayed.
Better no information than partial partial information.
Note: The package dependencies for 'libmyth-python' is handled
correctly by other means within each distribution, therefore
no information is lost.

Refs MythTV#392
@garybuhrmaster
Copy link
Contributor

V32 will only support python3.

Thanks for the confirmation.

rcrdnalor added a commit to rcrdnalor/mythtv that referenced this issue Oct 26, 2021
rcrdnalor added a commit that referenced this issue Nov 8, 2021
for package creation.

In Python 3.10 and 3.11, distutils will be formally marked as deprecated.
See https://www.python.org/dev/peps/pep-0632/
Albeit setuptools uses distutils internally, this will continue to work
in the future, because setuptools includes its own version of distutils.
See https://setuptools.pypa.io/en/latest/deprecated/index.html
Tested on python 3.6+.

Note: Each distribution patches heavily setuptools and distutils, the
package maintainers should check correctness of the new builds.

Although python PEP 518 proposes a new format to specify the build system
for python packages (pyproject.toml), this is not fully implemented in
python3.6 and can't be used.

Fixes #392
rcrdnalor added a commit that referenced this issue Nov 8, 2021
from 'setup.cfg' because it is not handled correctly by various
package managers, like Ubuntu. Only the last item is displayed.
Better no information than partial partial information.
Note: The package dependencies for 'libmyth-python' is handled
correctly by other means within each distribution, therefore
no information is lost.

Refs #392
rcrdnalor added a commit that referenced this issue Nov 8, 2021
if not given by make environment

Refs #392
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:bindings:python Python Bindings task:developer Developer Task
Development

Successfully merging a pull request may close this issue.

3 participants