Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dependencies: Add support for Python 3.12 #6161

Merged
merged 2 commits into from
Nov 16, 2023

Conversation

sphuber
Copy link
Contributor

@sphuber sphuber commented Oct 25, 2023

The file requirements/requirements-py-3.12.txt is added which provides
a complete environment for Python 3.12. The CI is updated to add Python
3.12 in all strategy matrices or replace Python 3.11 where only the
oldest and latest Python version are tested. Note that the Python
version for the pre-commit jobs are kept at 3.10 for now. The reason
is that in Python 3.12 f-strings are improved by allowing nested quotes.
For example:

f'some_dict['key']'

is now supported, whereas before Python 3.12 this would not work since
the nested quotes would not be parsed correctly and the internal quotes
had to be either escaped or changed for double quotes.

A number of dependencies had to be updated to make them compatible with
Python 3.12, usually because older version still relied on the
distutils and pkg_resources standard lib modules which have been
removed. The utils/dependency_management.py had to be updated similarly
to replace pkg_resources with packaging. The latter had to be
updated to packaging==23.0 in order to have the __eq__
implementation for the Requirement class which the script relies on.

The sphinxcontrib-details-directive dependency is removed. It was used
for the sphinx extension to add the ports of port namespaces in HTML's
<details> tags, allowing them to be collapsed. This could help with
readability in case of large namespaces. However, this package breaks on
Python 3.12 since it imports the deprecated pkg_resources package.
Since the package has not been maintained since 4 years, it is unlikely
this will be fixed it and so instead it is removed for now. See
https://github.com/sphinx-contrib/sphinxcontrib-details-directive

@sphuber sphuber force-pushed the feature/python-3.12 branch 4 times, most recently from 0c12356 to 51da042 Compare October 25, 2023 13:58
@danielhollas
Copy link
Collaborator

Wow, that was much faster than Python 3.11, cool!

@sphuber sphuber force-pushed the feature/python-3.12 branch 2 times, most recently from 63e8b39 to 2b10445 Compare October 25, 2023 14:38
@sphuber
Copy link
Contributor Author

sphuber commented Oct 25, 2023

Wow, that was much faster than Python 3.11, cool!

Well, we are not quite there yet 😉 the tests are actually hanging on Python 3.12 and we need the myst-nb package to make a release. But I am getting close.

@sphuber sphuber force-pushed the feature/python-3.12 branch 6 times, most recently from 973de81 to b5009f2 Compare November 16, 2023 09:00
The file `requirements/requirements-py-3.12.txt` is added which provides
a complete environment for Python 3.12. The CI is updated to add Python
3.12 in all strategy matrices or replace Python 3.11 where only the
oldest and latest Python version are tested. Note that the Python
version for the `pre-commit` jobs are kept at 3.10 for now. The reason
is that in Python 3.12 f-strings are improved by allowing nested quotes.
For example:

    f'some_dict['key']'

is now supported, whereas before Python 3.12 this would not work since
the nested quotes would not be parsed correctly and the internal quotes
had to be either escaped or changed for double quotes.

A number of dependencies had to be updated to make them compatible with
Python 3.12, usually because older version still relied on the
`distutils` and `pkg_resources` standard lib modules which have been
removed. The `utils/dependency_management.py` had to be updated similarly
to replace `pkg_resources` with `packaging`. The latter had to be
updated to `packaging==23.0` in order to have the `__eq__`
implementation for the `Requirement` class which the script relies on.

The memory leak tests are skipped on Python 3.12 because currently they
hang. The problem is with the `pympler.muppy.get_objects` method. This
calls `gc.collect` internally, but that call is blocking. The exact
cause is as of yet unknown. The garbage collecting has been changed in
Python 3.12 so it is not completely unexpected either.

The `sphinxcontrib-details-directive` dependency is removed. It was used
for the sphinx extension to add the ports of port namespaces in HTML's
`<details>` tags, allowing them to be collapsed. This could help with
readability in case of large namespaces. However, this package breaks on
Python 3.12 since it imports the deprecated `pkg_resources` package.
Since the package has not been maintained since 4 years, it is unlikely
this will be fixed it and so instead it is removed for now. See
https://github.com/sphinx-contrib/sphinxcontrib-details-directive
This dependency was temporarily removed since it didn't yet support
sqlalchemy v2, but that has now been released with `v0.2.0`.
@sphuber sphuber merged commit ca9000f into aiidateam:main Nov 16, 2023
38 checks passed
@sphuber sphuber deleted the feature/python-3.12 branch November 16, 2023 14:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants