Skip to content

Commit

Permalink
Shaves-off about 3 minutes from usage of ARM instances on CI (#24052)
Browse files Browse the repository at this point in the history
Preparing airflow packages and provider packages does not
need to be done on ARM and actually the ARM instance is idle
while they are prepared during cache building.

This change moves preparation of the packages to before
the ARM instance is started which saves about 3 minutes of ARM
instance time.

(cherry picked from commit 8804b1a)
  • Loading branch information
potiuk committed May 31, 2022
1 parent 81c099e commit 9eaa746
Showing 1 changed file with 12 additions and 12 deletions.
24 changes: 12 additions & 12 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1684,16 +1684,6 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
- run: ./scripts/ci/install_breeze.sh
- name: "Free space"
run: breeze free-space
- name: "Start ARM instance"
run: ./scripts/ci/images/ci_start_arm_instance_and_connect_to_docker.sh
if: matrix.platform == 'linux/arm64'
- name: "Push CI cache ${{ matrix.python-version }} ${{ matrix.platform }}"
run: >
breeze build-image
--prepare-buildx-cache
--platform ${{ matrix.platform }}
env:
PYTHON_MAJOR_MINOR_VERSION: ${{ matrix.python-version }}
- name: >
Pull CI image for PROD build
${{ env.PYTHON_MAJOR_MINOR_VERSION }}:${{ env.IMAGE_TAG_FOR_THE_BUILD }}"
Expand All @@ -1704,17 +1694,27 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
IMAGE_TAG: ${{ env.IMAGE_TAG_FOR_THE_BUILD }}
- name: "Cleanup dist and context file"
run: rm -fv ./dist/* ./docker-context-files/*
- name: "Prepare providers packages"
- name: "Prepare providers packages for PROD build"
run: >
breeze prepare-provider-packages
--package-list-file ./scripts/ci/installed_providers.txt
--package-format wheel
env:
VERSION_SUFFIX_FOR_PYPI: "dev0"
- name: "Prepare airflow package"
- name: "Prepare airflow package for PROD build"
run: breeze prepare-airflow-package --package-format wheel
env:
VERSION_SUFFIX_FOR_PYPI: "dev0"
- name: "Start ARM instance"
run: ./scripts/ci/images/ci_start_arm_instance_and_connect_to_docker.sh
if: matrix.platform == 'linux/arm64'
- name: "Push CI cache ${{ matrix.python-version }} ${{ matrix.platform }}"
run: >
breeze build-image
--prepare-buildx-cache
--platform ${{ matrix.platform }}
env:
PYTHON_MAJOR_MINOR_VERSION: ${{ matrix.python-version }}
- name: "Move dist packages to docker-context files"
run: mv -v ./dist/*.whl ./docker-context-files
- name: "Push PROD cache ${{ matrix.python-version }} ${{ matrix.platform }}"
Expand Down

0 comments on commit 9eaa746

Please sign in to comment.