Skip to content

Commit

Permalink
Add python upgrade guide (#1826)
Browse files Browse the repository at this point in the history
* Add python upgrade guide

* changes

* comments

* Update PYTHON_UPGRADE_GUIDE.MD
  • Loading branch information
atalman committed May 21, 2024
1 parent ea95ac3 commit 91eb7c4
Showing 1 changed file with 38 additions and 0 deletions.
38 changes: 38 additions & 0 deletions PYTHON_UPGRADE_GUIDE.MD
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Python Upgrade Runbook

So you wanna upgrade PyTorch to support a new Python? Follow these steps in order!

### 0.Before starting upgrade process. Check the package availability

Package availability to validate before starting upgrade process :

1) Manywheel build: Python version is avalilable in either of these sources:
- Github: https://github.com/python/cpython
- Python ftp[here](https://www.python.org/ftp/python/). Please consult the [devguide.python.org/versions](https://devguide.python.org/versions/)

2) Conda build: Python package is available via anaconda [here](https://anaconda.org/anaconda/python/files). The upgrade for manywheel binaries can continue before this requirement is met.

## 1. Maintain Progress and Updates
Make an issue to track the progress, for example CUDA issue [#56721: Support 11.3](https://github.com/pytorch/pytorch/issues/56721). This is especially important as many PyTorch external users are interested in CPython upgrades.

## 2. Modify scripts to install the new Python for Docker Manywheel images.

1. Follow this [PR 1427](https://github.com/pytorch/builder/pull/1427) with followup [PR 1568](https://github.com/pytorch/builder/pull/1568) for all steps in this section
2. Validate that Build manywheel docker images is successful [here](https://github.com/pytorch/builder/blob/main/.github/workflows/build-manywheel-images.yml)

## 3. Modify scripts to install the new Python for Core nightyly Manywheel and Conda builds.

1. Builder PRs: Follow this [PR 1586](https://github.com/pytorch/builder/pull/1586) and [PR 1593](https://github.com/pytorch/builder/pull/1593)
2. Pytorch PRs: [PR 112882](https://github.com/pytorch/pytorch/pull/112882)
3. Builder PR and Pytorch PR need to be tested togeather. After landing initiate nightly build and make sure nightly build is successful via [HUD](https://hud2.pytorch.org/hud/pytorch/pytorch/nightly/1?per_page=50)

## 4. Land Test-Infra PRs and make sure nighly domains builds are successful

1. Test-Infra PRs: [PR 4768](https://github.com/pytorch/test-infra/pull/4768), [PR 4843](https://github.com/pytorch/test-infra/pull/4843)
2. After landing it, intiate domain nightly builds and fix all follow up issues. Example vision [PR 8137](https://github.com/pytorch/vision/pull/8137)
3. At this point we should have complete set of nightly builds for pytorch, vision and audio. Please validate nightly binaries with [validate-binaries workflow](https://github.com/pytorch/builder/actions/workflows/validate-binaries.yml) to make sure smoke tests are passing for Python 3.12

## 5. Add support for new Pythron to CI

1. Pytorch PR: [PR 117853](https://github.com/pytorch/pytorch/pull/117853)
2. Please note, for any failures that can't be resolved easily, we ned to create new issue, and follow up on it similar to this [example](https://github.com/pytorch/pytorch/issues/120233)

0 comments on commit 91eb7c4

Please sign in to comment.