Skip to content

Commit

Permalink
Use PyPI constraints for PROD image in non-main branch (#33789)
Browse files Browse the repository at this point in the history
When we are building PROD image in CI for non main branch, we are
installing providers from PyPI rather than building them locally
from sources. Therefore we should use `PyPI` constraints for
such builds not the "source" constraints (they might differ).

This PR adds two steps:

* In the CI build, when we do not build providers we generate
  PyPI constraints additionally to source constraints
* In the PROD build we use the PyPI constraints in case we
  do not build providers locally
  • Loading branch information
potiuk committed Aug 27, 2023
1 parent 8227db3 commit f9276f0
Show file tree
Hide file tree
Showing 2 changed files with 18 additions and 2 deletions.
8 changes: 7 additions & 1 deletion .github/actions/build-ci-images/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,12 +34,18 @@ runs:
- name: "Build & Push AMD64 CI images ${{ env.IMAGE_TAG }} ${{ env.PYTHON_VERSIONS }}"
shell: bash
run: breeze ci-image build --push --tag-as-latest --run-in-parallel --upgrade-on-failure
- name: "Show dependencies to be upgraded"
- name: "Generate source constraints"
shell: bash
run: >
breeze release-management generate-constraints --run-in-parallel
--airflow-constraints-mode constraints-source-providers
if: env.UPGRADE_TO_NEWER_DEPENDENCIES != 'false'
- name: "Generate PyPI constraints"
shell: bash
run: >
breeze release-management generate-constraints --run-in-parallel
--airflow-constraints-mode constraints
if: env.UPGRADE_TO_NEWER_DEPENDENCIES != 'false' and ${{ inputs.build-provider-packages != 'true' }}
- name: "Print dependency upgrade summary"
shell: bash
run: |
Expand Down
12 changes: 11 additions & 1 deletion .github/actions/build-prod-images/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,14 +62,24 @@ runs:
name: constraints
path: ./docker-context-files
if: env.UPGRADE_TO_NEWER_DEPENDENCIES != 'false'
- name: "Build & Push PROD images ${{ env.IMAGE_TAG }}:${{ env.PYTHON_VERSIONS }}"
- name: "Build & Push PROD images with source providers ${{ env.IMAGE_TAG }}:${{ env.PYTHON_VERSIONS }}"
shell: bash
run: >
breeze prod-image build --tag-as-latest --run-in-parallel --push
--install-packages-from-context --airflow-constraints-mode constraints-source-providers
--use-constraints-for-context-packages
env:
COMMIT_SHA: ${{ github.sha }}
if: ${{ inputs.build-provider-packages == 'true' }}
- name: "Build & Push PROD images with PyPi providers ${{ env.IMAGE_TAG }}:${{ env.PYTHON_VERSIONS }}"
shell: bash
run: >
breeze prod-image build --tag-as-latest --run-in-parallel --push
--install-packages-from-context --airflow-constraints-mode constraints
--use-constraints-for-context-packages
env:
COMMIT_SHA: ${{ github.sha }}
if: ${{ inputs.build-provider-packages != 'true' }}
- name: "Fix ownership"
shell: bash
run: breeze ci fix-ownership
Expand Down

0 comments on commit f9276f0

Please sign in to comment.