Drop cpython#104135 sitecustomize into onedir Python during CI Deps#69490
Merged
dwoz merged 2 commits intoJun 19, 2026
Merged
Conversation
3006.x's Windows CI Deps job has continued failing on 3.14.6 (saltstack#69486 didn't help): the venv that pip runs in isn't created from the runner- host Python at all - noxfile.py's ci-test-onedir session uses the onedir's relenv-bundled Python (3.10.20 on 3006.x). Python 3.10's ssl._load_windows_store_certs concatenates every Windows-root-store cert and hands them to load_verify_locations as one blob; OpenSSL 3.5.x (relenv >= 0.22.13) rejects the whole blob on a single bad cert, so pip's TLS to pypi.org dies with [ASN1: NOT_ENOUGH_DATA] before any deps land. dwoz's salt/__init__.py monkey-patch (4d1e4db, e68db22, f222fe3) cannot help here - salt is not yet importable when pip first runs. Fix: ship the same iter-and-skip patch as cicd/windows-ssl-104135- sitecustomize.py and copy it into the onedir's Lib/site-packages between Decompress-Onedir-Tarball and Install-Nox in the Windows build-deps-ci job. With nox's --system-site-packages the venv inherits it, so the patch fires at every Python start-up before pip touches the network. Self-disables on Python 3.11+, which has the upstream fix in stdlib. Revert saltstack#69486's "3.14" -> "3.14.6" pin in layout.yml.jinja: that chose the runner-host Python, which never participates in the failing TLS - the pin was based on the wrong hypothesis and only added churn. Cross-link the three Python-3.10-only work-arounds in their comments so a future forward-merge drops them as a unit on any branch whose onedir Python is >= 3.11.
dwoz
added a commit
that referenced
this pull request
Jun 19, 2026
…#104135 After #69490 landed, the `ci-test-onedir` job got past Install Deps, but the next session - `pre-archive-cleanup` (noxfile.py:1397-1402) - failed at the same step with the same `[ASN1: NOT_ENOUGH_DATA]`. That session also uses the onedir Python but creates its virtualenv WITHOUT `--system-site-packages`, so the `sitecustomize.py` we dropped in `artifacts/salt/Lib/site-packages/` isn't loaded by it and the buggy `_load_windows_store_certs` is back. Patch `Lib/ssl.py` in the onedir itself instead: virtualenv's venvs always read stdlib from the base Python's `Lib/` (with or without `--system-site-packages`), so a `Lib/ssl.py` append covers every venv launched from the onedir, plus the onedir Python directly. Verified on a local relenv 0.22.14 build (Python 3.10.20, OpenSSL 3.5.7): both a `--system-site-packages` venv and an isolated venv now report `_salt_safe_load_windows_store_certs` as the qualname for `ssl.SSLContext._load_windows_store_certs`, and a synthetic truncated cert injected via a monkey-patched `_ssl.enum_certificates` no longer aborts `load_default_certs()`. Drops `cicd/windows-ssl-104135-sitecustomize.py` (now redundant) and adds `cicd/windows-ssl-104135-patch.py` (idempotent in-place patcher; takes the ssl.py path as argv). The CI step swaps `cp ... -> sitecustomize.py` for `python3 cicd/windows-ssl-104135-patch.py artifacts/salt/Lib/ssl.py`. Cross-references in `salt/__init__.py` and `salt/ext/tornado/netutil.py` updated to point at the new filename so the trio still drops together when this branch's onedir Python moves off 3.10.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
#69486's "3.14" → "3.14.6" pin did not fix the Windows nightlies (run 27799109831 still fails the same way) - the venv that pip is running in isn't created from the runner-host Python at all.
noxfile.py:1182defines theci-test-onedirsession aspython=str(ONEDIR_PYTHON_PATH), so the venv's interpreter is the salt onedir's relenv-bundled Python - 3.10.20 on 3006.x. Python 3.10's stdlibssl._load_windows_store_certsconcatenates every cert in the Windows root store and feeds them toload_verify_locations(cadata=...)as one blob, which OpenSSL 3.5.x (relenv ≥ 0.22.13) refuses to parse if any single cert is ASN.1-malformed - exactly cpython#104135. pip's first HTTPS GET to pypi.org dies with[ASN1: NOT_ENOUGH_DATA]and the dep install never starts.dwoz's
salt/__init__.pymonkey-patch (`4d1e4db01a`, `e68db2220d`, `f222fe37e7`) fires correctly at salt-runtime but cannot help during build-deps-ci: salt is not yet installed when pip first runs in the new venv.Fix
Ship the iter-and-skip patch as
cicd/windows-ssl-104135-sitecustomize.pyand add a workflow step in the Windows job ofbuild-deps-ci-action.ymlthat copies it toartifacts/salt/Lib/site-packages/sitecustomize.pybetween Decompress Onedir Tarball and Install Nox. Because theci-test-onedirnox session creates its venv with--system-site-packages(noxfile.py:1184), the venv inherits the onedir's site-packages directory, so Python loads the sitecustomize at every interpreter start - including the pip invocation that's been failing - before any TLS code runs.The new file self-disables on Python 3.11+ (whose stdlib already has the upstream iter-and-skip fix), so it's safe to forward-merge as a no-op into branches whose onedir uses Python 3.14.
Also reverts the "3.14" → "3.14.6" pin from #69486 - that targeted the wrong Python and only adds churn.
Cleanup discipline
All three Python-3.10-only work-arounds now cross-reference each other in their comments:
salt/__init__.py(salt-runtime monkey-patch)salt/ext/tornado/netutil.py(certifi pin on Windows)cicd/windows-ssl-104135-sitecustomize.py+ the Apply-sitecustomize step inbuild-deps-ci-action.ymlWhen 3006.x's onedir eventually moves off Python 3.10 (or these get forward-merged to a branch whose onedir Python is ≥ 3.11), drop them as a unit.
Test plan
CI Deps / Windows (amd64)andCI Deps / Windows (x86)go green on this branch's CI run.