Windows Python downloads are unreliable

[thewtex]

e.g.

Mode                 LastWriteTime         Length Name
26
----                 -------------         ------ ----
27
d----           1/31/2023  6:36 PM                Downloads
28
Download install-utils.ps1
29
Installing Python version 310
30
Installing Python for architecture x64
31
Defaulting 'pythonPrependPath' variable to 0.
32
Skipping 36
33
Skipping 37
34
Skipping 38
35
Skipping 39
36
WARNING: Unable to set PowerShell to use TLS 1.2 and TLS 1.1 due to old .NET Framework installed. If you see underlying connection closed or trust errors, you may need to do one or more of the following: (1) upgrade to .NET Framework 4.5 and PowerShell v3, (2) specify internal Chocolatey package location (set $env:chocolateyDownloadUrl prior to install or host the package internally), (3) use the Download + PowerShell method of install. See https://chocolatey.org/install for all install options.
37
Download https://www.python.org/ftp/python/3.10.8/python-3.10.8-amd64.exe
38
WARNING: Unable to set PowerShell to use TLS 1.2 and TLS 1.1 due to old .NET Framework installed. If you see underlying connection closed or trust errors, you may need to do one or more of the following: (1) upgrade to .NET Framework 4.5 and PowerShell v3, (2) specify internal Chocolatey package location (set $env:chocolateyDownloadUrl prior to install or host the package internally), (3) use the Download + PowerShell method of install. See https://chocolatey.org/install for all install options.
39
Download https://www.python.org/ftp/python/3.10.9/python-3.10.9-amd64.exe
40
Write-Error: D:\a\_temp\e62e2111-1b4c-4012-9a1e-56c17d13c5a3.ps1:12
41
Line |
42
  12 |  ./windows-download-cache-and-build-module-wheels.ps1 "10"
43
     |  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
44
     | The SSL connection could not be established, see inner exception.

https://github.com/InsightSoftwareConsortium/itk-wasm/actions/runs/4053853776/jobs/6982667101

[tbirdso]

@thewtex Do you have thoughts on what we could do to address this? If the issue is with the python.org or GitHub servers I am not sure what we can do other than introducing retries.

[jcfr]

Wondering if it could be related to: https://scikit-ci-addons.readthedocs.io/en/latest/addons.html#addressing-underlying-connection-closed

[jcfr]

To identify potential issue with a domain, this tool may be useful. See https://www.ssllabs.com/ssltest/

[tbirdso]

Thanks @jcfr , this looks like a good path for investigation.

[tbirdso]

A fix is introduced in scikit-build/scikit-ci-addons#105 to use TLS 1.2 and 1.1 for downloading GitHub scripts and Python executables.

Immediately following the patch SSL errors have continued to be observed in itk-wasm and other external modules despite fetching scripts from scikit-build/scikit-ci-addons/master. https://github.com/InsightSoftwareConsortium/itk-wasm/actions/runs/4140383502/jobs/7158987740

I am convinced that the issue lies in GitHub servers caching scripts or otherwise being out of sync. I am able to reproduce in a development branch where a CI runner fetches an old script from the master branch but clicking on the URL in the log returns the updated script. See the "Build Python Package" section of the log and compare install_utils.ps1 from the URL with the printout of the downloaded script in the log: https://github.com/InsightSoftwareConsortium/itk-wasm/actions/runs/4144801556/jobs/7168313166

@thewtex Do you have any thoughts on what can be done to address this? If it is indeed a GitHub issue it may resolve by itself if there is a periodic sync.

[tbirdso]

@thewtex It looks like scikit-ci-addons script links on raw.githubusercontent.com synced over the weekend and itk-wasm builds are no longer encountering Windows download issues. Are you still seeing TLS concerns or is this issue ready to be closed?

[thewtex]

@tbirdso I am seeing consistently better behavior now, thank you!!

I am now getting InsightSoftwareConsortium/ITKPythonPackage#251 , which is a separate issue, but could you please take a look.