Add Python 3.9

[hugovk]

Tool information

• Tool name: Python 3.9.0 final
• Tool license: Python Software Foundation License docs.python.org/3/license.html
• Add or update? Add
• Desired version: 3.9.0, and latest 3.9.x when available
• Approximate size:
• If this is an add request:
  • Brief description of tool: The newest version of Python
  • URL for tool's homepage: https://www.python.org/download/pre-releases/
  • Provide a basic test case to validate the tool's functionality: python -c "import sys; assert sys.version_info[:3] == (3, 9, 0)"

Area for Triage:

• valueFor: 'Area for Triage'
  contains: 'Python'
  addLabels: ['Area: Python']

Question, Bug, or Feature?:

Feature

Virtual environments affected

• macOS 10.15
• Ubuntu 16.04 LTS
• Ubuntu 18.04 LTS
• Ubuntu 20.04 LTS
• Windows Server 2016 R2
• Windows Server 2019

Can this tool be installed during the build?

#912 said:

Only on Windows via a Nuget package. You can build from source during build for other platforms, which I assume only takes 3-4 minutes (based on our CI running on the same images).

Tool installation time in runtime

#912 said:

Should be under 1 minute

Are you willing to submit a PR?

Yes, I can give it a go, with pointers.

PS It took 24 days for 3.8: https://github.com/microsoft/azure-pipelines-image-generation/issues/1317 :)

[dibir-magomedsaygitov]

Hello, @hugovk!
Thank you for your request. We are already working on adding python 3.9 on our images. I will post update message when python 3.9 is available to use.

[maxim-lobanov]

Hello everyone, Python 3.9 is already available on GitHub Actions via setup-python task (It is not pre-cached on image yet, so it will be installed on-flight with minor latency):

- uses: actions/setup-python@v2
  with:
    python-version: '3.9'

We also going to deploy new image with pre-cached Python 3.9 next week.

[hugovk]

Confirmed it's now available on GHA, thank you!

Took ~10s to install on Linux, ~60s on macOS and ~51s on Windows, not a problem for me, and good to hear the pre-cached images is coming soon.

https://github.com/hugovk/test/runs/1215290584?check_suite_focus=true

Run actions/setup-python@v2
Version 3.9 was not found in the local cache
Version 3.9 is available for downloading
Download from "actions/python-versions@3.9.0-83838 python-3.9.0-linux-20.04-x64.tar.gz (download)"
Extract downloaded archive
/usr/bin/tar xz --warning=no-unknown-keyword -C /home/runner/work/_temp/e8e512d5-ecbb-4563-9374-a1796267419e -f /home/runner/work/_temp/e9bdf138-7f69-49ec-8166-59a47b1d3ef4
Execute installation script
Check if Python hostedtoolcache folder exist...
Create Python 3.9.0 folder
Copy Python binaries to hostedtoolcache folder
Create additional symlinks (Required for the UsePythonVersion Azure Pipelines task and the setup-python GitHub Action)
Upgrading PIP...
Looking in links: /tmp/tmpiitvwvjj
Requirement already satisfied: setuptools in /opt/hostedtoolcache/Python/3.9.0/x64/lib/python3.9/site-packages (49.2.1)
Requirement already satisfied: pip in /opt/hostedtoolcache/Python/3.9.0/x64/lib/python3.9/site-packages (20.2.3)
Collecting pip
Downloading pip-20.2.3-py2.py3-none-any.whl (1.5 MB)
Installing collected packages: pip
Successfully installed pip-20.2.3
Create complete file
Successfully setup CPython (3.9.0)

[charris]

Commenting for followup postings.

[hugovk]

@charris tip for next time: no need to comment to subscribe to followup postings, there's a Subscribe button, which also avoids notifying dozens of issue and repo watchers :)

[laike9m]

Hello everyone, Python 3.9 is already available on GitHub Actions via setup-python task (It is not pre-cached on image yet, so it will be installed on-flight with minor latency):

- uses: actions/setup-python@v2
  with:
    python-version: '3.9'

We also going to deploy new image with pre-cached Python 3.9 next week.

Hi Maxim, what about Azure pipelines?

[maxim-lobanov]

@laike9m , Unfortunately, Azure Pipelines UsePython task can't install versions on-flight like setup-python so we have to wait for next week until new image with Python 3.9 is deployed.
(I suggest to upvote feature request for UsePython to work the same way like setup-python)

[laike9m]

@maxim-lobanov Got it. Upvoted.

[webknjaz]

@maxim-lobanov All my jobs hit API rate limit when I set them to 3.9 :(

Is there a problem with it? https://github.com/ansible/pylibssh/pull/152/checks?check_run_id=1232497424

[charris]

Any update on the Windows availability timeline?

[miketimofeev]

@charris we're going to finish windows images deployment tomorrow.

[laike9m]

BTW it seems there's some problem with Mac + Python 3.9:
https://dev.azure.com/laike9m/laike9m/_build/results?buildId=315&view=logs&j=0ff66468-69a5-508b-6ca1-fbfd9e851960&t=6505030a-b067-588a-292f-b15686a87159

[remyjette]

@laike9m That looks like an issue with your script, which is trying to use add-apt-repository and apt-get on a macOS agent.

As mentioned above, Python 3.9 is already available on the macOS agents. Use the UsePythonVersion task to add the binaries to the PATH.

I've been using Python 3.9 on the macOS and Ubuntu Azure Pipelines agents with no issues.

[hugovk]

🎉 The rollout progress is now 100% for Windows at https://github.com/actions/virtual-environments#available-environments:

Does this mean Python 3.9 is now available everywhere on Azure Pipelines?

[hugovk]

PS It took 24 days for 3.8: microsoft/azure-pipelines-image-generation#1317 :)

This year it took 22 days! 🎉

• https://twitter.com/hugovk/status/1321536957965574151

[mattpackwood]

Just adding myself so I can monitor...

[ThiefMaster]

@mattpackwood: There's a "subscribe" button on the side ;)

[mattpackwood]

@ThiefMaster Duh, sorry!

[MaksimZhukov]

Hello everyone!
Python 3.9 has been successfully deployed to the all images and currently is available on the Windows 2019 and 2016 as well. Thanks for you patience! I'm closing this issue, but feel free to contact us if you have any concerns.

[laike9m]

Thanks for the hard work on this.

[joaander]

@MaksimZhukov Some of my pipelines requesting Python 3.9 on windows-2019 randomly fail (https://dev.azure.com/glotzerlab/gsd/_build/results?buildId=6977&view=logs&j=45d08669-c843-5a21-c5fb-d107fba53589&t=4b9eac60-fda1-5fb4-16ab-3474dd2a30bb&l=9). When will this change be deployed to all windows hosts?

[maxim-lobanov]

@joaander , Hm, that deployment has been finished a long time ago (and a few more deployments after that). But it looks like some pools were not updated. We will take care about fixing it
Doesn't you mind to create the separate issue to track it? (it is not the issue with Python itself, it is more about the fact that some builds are run on outdated image)

[joaander]

@joaander , Hm, that deployment has been finished a long time ago (and a few more deployments after that). But it looks like some pools were not updated. We will take care about fixing it
Doesn't you mind to create the separate issue to track it? (it is not the issue with Python itself, it is more about the fact that some builds are run on outdated image)

Certainly: Open in #2039