Add local airflow + configuration cleanup#37644
Conversation
Our CI sometimes reuses same runner for several jobs (this is until we switch to --ephemeral runners) and this might lead to some side-effects between runners. When the two runners are run for different branches, and airflow is locally installed, it might lead to configuration being written by one version and used by another. This - for example - made https://github.com/apache/airflow/actions/runs/8011411765/job/21884630103 to fail. This PR adds extra cleanup to `breeze ci free-space` command and add extra step in `breeze cleanup` to remove installed airflow and the whole airflow home.
|
I used breeze cleanup just now and realized that it deletes everything under the airflow path. Since this also includes the repo .git folder this could be risky for developers if they expect it to clean up docker and build related stuff. |
|
It's a very bad idea to checkout airflow in |
|
Or so I thought: PR adding protection against such checkout is here: #37697 After it's merged, when you checkout your Airlfow in HOME/airflow and use breeze you will see this (and you won't be able to run breeze)
:
|
|
Ok thanks I wasn't aware of this |
So now it will be clear to anyone who is not aware :) |

Our CI sometimes reuses same runner for several jobs (this is until we switch to --ephemeral runners) and this might lead to some side-effects between runners. When the two runners are run for different branches, and airflow is locally installed, it might lead to configuration being written by one version and used by another. This - for example - made
https://github.com/apache/airflow/actions/runs/8011411765/job/21884630103 to fail.
This PR adds extra cleanup to
breeze ci free-spacecommand and add extra step inbreeze cleanupto remove installed airflow and the whole airflow home.^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.