Release v0.3.5 #1044
Merged
Release v0.3.5 #1044
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Prepare 0.3.4
To do so the test suite allows a weak cryptographic alogorithm (SHA1) to be used, principally for CentOS 6 targets. This can be removed if/when support for older (legacy) targets is dropped. Only the test suite enables this known weak alogorithm. Mitogen as-shipped doesn't enable or disable algorithms.
tests: Support Ubuntu 22.04 as test suite runner (controller)
Fixes
```
======================================================================
ERROR: setUpClass (ssh_test.BannerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/vsts/work/1/s/tests/testlib.py", line 625, in setUpClass
cls.dockerized_ssh = DockerizedSshDaemon(**daemon_args)
File "/home/vsts/work/1/s/tests/testlib.py", line 553, in __init__
self.start_container()
File "/home/vsts/work/1/s/tests/testlib.py", line 533, in start_container
self._get_container_port()
File "/home/vsts/work/1/s/tests/testlib.py", line 510, in _get_container_port
self.port = int(bport)
ValueError: invalid literal for int() with base 10: ':]:32770'
```
Preperation for Python 3.11 support
This is how the package documentation recommends and it's less likely to interfere with new features in stdlib subprocess module.
Prepare for Python 3.11
Co-authored-by: Alex Willmer <alex@moreati.org.uk>
Add Python 3.11 support
Bare minimum syntax errors and requirements constraints to work with Netlify hosting.
Found with sphinx-build -b linkcheck. Not all flagged URLs have been changed, e.g. Ansible plugins, deleted Github users.
docs: Fix generating static website on Netlify
This should address the warning in Azure Pipelines > You should provide GitHub token if you want to download a python release. > Otherwise you may hit the GitHub anonymous download limit. The token is provided from a secret variable in the pipeline.
ci: Authenticate UsePythonVersion requests to Github
This prevents unreleased versions appearing on the website (e.g. 0.3.5.dev0), but introduces the risk of forgetting to update the website after a release. A better fix requires deeper design/workflow thought. refs mitogen-hq#1028
With current macOS 11 runner images (20231216.1) the `python` on `$PATH` is
Python 3.12 and setuptools isn't installed by default. E.g.
```
python -mtox -e "py27-mode_localhost-ansible4"
========================== Starting Command Output ===========================
/bin/bash --noprofile --norc
/Users/runner/work/_temp/93a29c4c-f606-45e4-8dbd-a4a5f51b8730.sh
GLOB sdist-make: /Users/runner/work/1/s/setup.py
ERROR: invocation failed (exit code 1), logfile:
/Users/runner/work/1/s/.tox/log/GLOB-0.log
================================== log start
===================================
Traceback (most recent call last):
File "/Users/runner/work/1/s/setup.py", line 32, in <module>
from setuptools import find_packages, setup
ModuleNotFoundError: No module named 'setuptools'
```
Installing setuptools under Python 3.12 chooses package versions incompatible
with Python 2.7. Additionally Mitogen isn't yet compatible with Python 3.12
(mitogen-hq#1033), so tests that call a local context with `python` fail.
With ansible/ansible#76677 Ansible fixed shebang substitution for Ansible modules and tightened up what shebang is allowed. Changing these fixes the tests using them with vanilla Ansible. https://docs.ansible.com/ansible/latest/dev_guide/testing/sanity/shebang.html
Task " Install slow profile for one account" removed because it duplicates earlier work.
…tests This has been lurking for years, raising it's head at unpredictable times. This change doesn't fix it, but it should make it a lot less mysterious.
Works with the VS Code modeline extension. Enables syntax highlighting.
Using https:// requires certificate store management and additional parameter passing that changed across Ansible and Python versions. Using http:// allows the same tests to be used across wider spans of Python version on the controller, and Python verison on the targets. Python 3.12 on a target + get_uri needs Ansible >= 8 (ansible-core >= 2.15). Python 3.12 removed deprecated httplib.HTTPSConnection() arguments. ansible/ansible#80751
… tests on macOS Should account for fiddling in mitogen.parent.Connection._first_stage() and symlinks. I won't be surprised if it breaks again soon and often.
cwd_show was useful when debugging these tests, worth keeping around.
The ini inventory parser doesn't support comments after a value, so the value parsed was "python3000 # Not expected to exist".
…eadable
On macOS when using a become plugin as an unprivileged user, to another
unprivileged user it is likely that the current working directory can't be
read. In this case os.cwd() raises PermissionError.
On versions of Python currently in the wild (March 2024, CPython <= 3.13) if
any non-builtin or non-frozen module (e.g. zlib, base64) is imported then
`importlib._bootstrap_external.PathFinder._path_importer_cache()` attempts to
call os.cwd() without catching PermissionError.
The previous comment about needing an extra .encode() appears to be wrong,
atleast for Python 3.x >= 3.6.
Command size increased by 54 bytes, bootstrap by 804 bytes. Changed from
codecs module to binascii & zlib because they're extensions, and importing
them triggers fewer supporting imports (e.g. encodings module).
Before
```
✗ ./preamble_size.py
SSH command size: 705
Bootstrap (mitogen.core) size: 17078 (16.68KiB)
Original Minimized Compressed
mitogen.parent 97884 95.6KiB 50515 49.3KiB 51.6% 12727 12.4KiB
13.0%
mitogen.fork 8436 8.2KiB 4130 4.0KiB 49.0% 1648 1.6KiB
19.5%
mitogen.ssh 10892 10.6KiB 6952 6.8KiB 63.8% 2113 2.1KiB
19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB
18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB
7.8%
mitogen.service 41699 40.7KiB 22477 22.0KiB 53.9% 5885 5.7KiB
14.1%
mitogen.fakessh 15577 15.2KiB 7989 7.8KiB 51.3% 2623 2.6KiB
16.8%
mitogen.master 51398 50.2KiB 25715 25.1KiB 50.0% 6886 6.7KiB
13.4%
```
After
```
✗ ./preamble_size.py
SSH command size: 759
Bootstrap (mitogen.core) size: 17882 (17.46KiB)
Original Minimized Compressed
mitogen.parent 98173 95.9KiB 50571 49.4KiB 51.5% 12747 12.4KiB
13.0%
mitogen.fork 8436 8.2KiB 4130 4.0KiB 49.0% 1648 1.6KiB
19.5%
mitogen.ssh 10892 10.6KiB 6952 6.8KiB 63.8% 2113 2.1KiB
19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB
18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB
7.8%
mitogen.service 41699 40.7KiB 22477 22.0KiB 53.9% 5885 5.7KiB
14.1%
mitogen.fakessh 15577 15.2KiB 7989 7.8KiB 51.3% 2623 2.6KiB
16.8%
mitogen.master 56116 54.8KiB 29427 28.7KiB 52.4% 7627 7.4KiB
13.6%
```
Fixes mitogen-hq#885
Refs python/cpython#115911
importlib.machinery.ModuleSpec and find_spec() were introduced in Python 3.4 under PEP 451. They replace the find_module() API of PEP 302, which was deprecated from Python 3.4. They were removed in Python 3.12 along with the imp module. This change adds support for the PEP 451 APIs. Mitogen should no longer import imp on Python versions that support ModuleSpec. Tests have been added to cover the new APIs. CI jobs have been added to cover Python 3.x on macOS. Refs mitogen-hq#1033 Co-authored-by: Witold Baryluk <witold.baryluk@gmail.com>
Most of the necessary changes were made in recent PEP 451 commits. This bumps the CI jobs, and declares the support. Test dependendancies are bumped to latest supportted/available versions. refs mitogen-hq#1033
Python 3.12 support
dw
reviewed
Mar 26, 2024
dw
approved these changes
Mar 26, 2024
moreati
commented
Mar 26, 2024
(cherry picked from commit 50efa53)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.