Skip to content

Fix GPU CI so that it runs on Python 3.8 #2973

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Fix GPU CI so that it runs on Python 3.8 #2973

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -455,6 +455,7 @@ jobs:
resource_class: gpu.small
environment:
image_name: "pytorch/manylinux-cuda101"
PYTHON_VERSION: << parameters.python_version >>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was working on the same issue on torchaudio, and although I understand that this solves the issue, I wonder if there is a better way.
The problem here is environment: image_name: "..." overwrites the environment defined in binary_common. It seems to me that the ideal implementation is that image_name should be appended to the set of environment key-value pairs defined in binary_common, so that the downstream jobs do not have to re-define them. But with quick searching on YAML syntax, I cannot find such a way.
On the other hand, binary_common was defined for binary build process (probably by @seemethere ) and I just re-used it in unit test jobs without knowing much about it in the initial implementation of CCI job configurations (in torchaudio, which was later ported to torchvision).
Environment values like UPLOAD_CHANNEL and PYTORCH_VERSION have become an essential part of unit test scripts in the previous release work, #2846, so ideally we want environment variables to be defined in one place and we want them to be automagically propagated.
The use of Docker adds extra layer to this as it seems inevitable to manually pass environment values to the container which runs the test.

steps:
- checkout
- designate_upload_channel
Expand All @@ -469,7 +470,7 @@ jobs:

- run:
name: Setup
command: docker run -t --gpus all -v $PWD:$PWD -w $PWD "${image_name}" .circleci/unittest/linux/scripts/setup_env.sh
command: docker run -e PYTHON_VERSION -t --gpus all -v $PWD:$PWD -w $PWD "${image_name}" .circleci/unittest/linux/scripts/setup_env.sh
- save_cache:

key: env-v2-linux-{{ arch }}-py<< parameters.python_version >>-{{ checksum ".circleci/unittest/linux/scripts/environment.yml" }}-{{ checksum ".circleci-weekly" }}
Expand Down Expand Up @@ -533,6 +534,7 @@ jobs:
name: windows-gpu
environment:
CUDA_VERSION: "10.1"
PYTHON_VERSION: << parameters.python_version >>
steps:
- checkout
- designate_upload_channel
Expand Down
4 changes: 3 additions & 1 deletion .circleci/config.yml.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -455,6 +455,7 @@ jobs:
resource_class: gpu.small
environment:
image_name: "pytorch/manylinux-cuda101"
PYTHON_VERSION: << parameters.python_version >>
steps:
- checkout
- designate_upload_channel
Expand All @@ -469,7 +470,7 @@ jobs:
{% endraw %}
- run:
name: Setup
command: docker run -t --gpus all -v $PWD:$PWD -w $PWD "${image_name}" .circleci/unittest/linux/scripts/setup_env.sh
command: docker run -e PYTHON_VERSION -t --gpus all -v $PWD:$PWD -w $PWD "${image_name}" .circleci/unittest/linux/scripts/setup_env.sh
- save_cache:
{% raw %}
key: env-v2-linux-{{ arch }}-py<< parameters.python_version >>-{{ checksum ".circleci/unittest/linux/scripts/environment.yml" }}-{{ checksum ".circleci-weekly" }}
Expand Down Expand Up @@ -533,6 +534,7 @@ jobs:
name: windows-gpu
environment:
CUDA_VERSION: "10.1"
PYTHON_VERSION: << parameters.python_version >>
steps:
- checkout
- designate_upload_channel
Expand Down