Skip to content
Permalink
Browse files
Convert selective checks to Breeze Python (#24610)
Instead of bash-based, complex logic script to perform PR selective
checks we now integrated the whole logic into Breeze Python code.

It is now much simplified, when it comes to algorithm. We've
implemented simple rule-based decision tree. The rules describing
the decision tree are now are now much easier
to reason about and they correspond one-to-one with the rules
that are implemented in the code in rather straightforward way.

The code is much simpler and diagnostics of the selective checks
has also been vastly improved:

* The rule engine displays status of applying each rule and
  explains (with yellow warning message what decision was made
  and why. Informative messages are printed showing the resulting
  output

* List of files impacting the decision are also displayed

* The names of "ci file group" and "test type" were aligned

* Unit tests covering wide range of cases are added. Each test
  describes what is the case they demonstrate

* `breeze selective-checks` command that is used in CI can also
  be used locally by just providing commit-ish reference of the
  commit to check. This way you can very easily debug problems and
  fix them

Fixes: #19971
  • Loading branch information
potiuk committed Jun 25, 2022
1 parent 4923bd1 commit d7bd72f494e7debec11672eeddf2e6ba5ef75fac
Show file tree
Hide file tree
Showing 22 changed files with 1,814 additions and 1,337 deletions.
@@ -109,18 +109,20 @@ jobs:
with:
persist-credentials: false
submodules: recursive
- name: "Setup python"
uses: actions/setup-python@v2
with:
# We do not have output from selective checks yet, so we need to hardcode python
python-version: 3.7
cache: 'pip'
cache-dependency-path: ./dev/breeze/setup*
- run: ./scripts/ci/install_breeze.sh
- name: Selective checks
id: selective-checks
env:
PR_LABELS: ${{ steps.get-latest-pr-labels.outputs.pullRequestLabels }}
run: |
if [[ ${GITHUB_EVENT_NAME} == "pull_request_target" ]]; then
# Run selective checks
./scripts/ci/selective_ci_checks.sh "${TARGET_COMMIT_SHA}"
else
# Run all checks
./scripts/ci/selective_ci_checks.sh
fi
PR_LABELS: "$${{ steps.get-latest-pr-labels.outputs.pullRequestLabels }}"
COMMIT_REF: "${{ github.sha }}"
run: breeze selective-check
- name: Compute dynamic outputs
id: dynamic-outputs
run: |
@@ -192,18 +192,20 @@ jobs:
fetch-depth: 2
persist-credentials: false
if: github.event_name == 'pull_request'
- name: "Setup python"
uses: actions/setup-python@v2
with:
# We do not have output from selective checks yet, so we need to hardcode python
python-version: 3.7
cache: 'pip'
cache-dependency-path: ./dev/breeze/setup*
- run: ./scripts/ci/install_breeze.sh
- name: Selective checks
id: selective-checks
env:
PR_LABELS: "${{ steps.source-run-info.outputs.pullRequestLabels }}"
run: |
if [[ ${GITHUB_EVENT_NAME} == "pull_request" ]]; then
# Run selective checks
./scripts/ci/selective_ci_checks.sh "${GITHUB_SHA}"
else
# Run all checks
./scripts/ci/selective_ci_checks.sh
fi
COMMIT_REF: "${{ github.sha }}"
run: breeze selective-check
# Avoid having to specify the runs-on logic every time. We use the custom
# env var AIRFLOW_SELF_HOSTED_RUNNER set only on our runners, but never
# on the public runners
@@ -43,19 +43,19 @@ jobs:
with:
fetch-depth: 2
persist-credentials: false
- name: "Setup python"
uses: actions/setup-python@v2
with:
# We do not have output from selective checks yet, so we need to hardcode python
python-version: 3.7
cache: 'pip'
cache-dependency-path: ./dev/breeze/setup*
- run: ./scripts/ci/install_breeze.sh
- name: Selective checks
id: selective-checks
env:
EVENT_NAME: ${{ github.event_name }}
TARGET_COMMIT_SHA: ${{ github.sha }}
run: |
if [[ ${EVENT_NAME} == "pull_request" ]]; then
# Run selective checks
./scripts/ci/selective_ci_checks.sh "${TARGET_COMMIT_SHA}"
else
# Run all checks
./scripts/ci/selective_ci_checks.sh
fi
COMMIT_REF: "${{ github.sha }}"
run: breeze selective-check

analyze:
name: Analyze
@@ -565,13 +565,17 @@ Configuration and maintenance
* Cleanup breeze with ``breeze cleanup`` command
* Self-upgrade breeze with ``breeze self-upgrade`` command
* Setup autocomplete for Breeze with ``breeze setup-autocomplete`` command
* Checking available resources for docker with ``breeze resource-check`` command
* Freeing space needed to run CI tests with ``breeze free-space`` command
* Fixing ownership of files in your repository with ``breeze fix-ownership`` command
* Print Breeze version with ``breeze version`` command
* Outputs hash of commands defined by ``breeze`` with ``command-hash-export`` (useful to avoid needless
regeneration of Breeze images)

CI tasks
--------
* Freeing space needed to run CI tests with ``breeze free-space`` command
* Fixing ownership of files in your repository with ``breeze fix-ownership`` command
* Checking available resources for docker with ``breeze resource-check`` command
* Deciding which tests should be run with ``breeze selective-check`` command

Release tasks
-------------

@@ -1295,8 +1299,8 @@ command but it is very similar to current ``breeze`` command):
</a>
</div>

Resource check
==============
Running resource check
----------------------

Breeze requires certain resources to be available - disk, memory, CPU. When you enter Breeze's shell,
the resources are checked and information if there is enough resources is displayed. However you can
@@ -1310,7 +1314,7 @@ Those are all available flags of ``resource-check`` command:


Freeing the space
=================
-----------------

When our CI runs a job, it needs all memory and disk it can have. We have a Breeze command that frees
the memory and disk space used. You can also use it clear space locally but it performs a few operations
@@ -1323,8 +1327,26 @@ Those are all available flags of ``free-space`` command:
:alt: Breeze free-space


Selective check
---------------

When our CI runs a job, it needs to decide which tests to run, whether to build images and how much the test
should be run on multiple combinations of Python, Kubernetes, Backend versions. In order to optimize time
needed to run the CI Builds. You can also use the tool to test what tests will be run when you provide
a specific commit that Breeze should run the tests on.

More details about the algorithm used to pick the right tests can be
found in `Selective Checks <SELECTIVE_CHECKS.md>`_.

Those are all available flags of ``selective-check`` command:

.. image:: ./images/breeze/output-selective-check.svg
:width: 100%
:alt: Breeze selective-check


Tracking backtracking issues for CI builds
==========================================
------------------------------------------

When our CI runs a job, we automatically upgrade our dependencies in the ``main`` build. However, this might
lead to conflicts and ``pip`` backtracking for a long time (possibly forever) for dependency resolution.

This file was deleted.

@@ -52,6 +52,6 @@ PLEASE DO NOT MODIFY THE HASH BELOW! IT IS AUTOMATICALLY UPDATED BY PRE-COMMIT.

---------------------------------------------------------------------------------------------------------

Package config hash: a80a853b2c32c284a68ccd6d468804b892a69f14d2ad1886bdaa892755cf6262660e2b9fc582bcae27ae478910055267a76edea2df658196198a0365150e93e5
Package config hash: 7279229e03b197f2bbd10ebb7b313f67bba3a704735d3688652efc5bdc1b3a60f2d1e0a144c89a2ecd11268b06888c5302a8774a8f392dc383bb940c99521db3

---------------------------------------------------------------------------------------------------------

0 comments on commit d7bd72f

Please sign in to comment.