Raise error when DagRun fails while running dag test
#36517
Conversation
kaxil
force-pushed
the
raise-error-dag-test
branch
2 times, most recently
from
8919958
to
3df3eba
Compare
I agree with @jscheffl. The default behavior of the cli command should be the status of the DAG run/runs. |
|
Once concern is backwards compatibility (and I think that's where @kaxil's solution came from). But yeah, I think you are right @jscheffl @dirrao - probably return value is not teally compatibility promise (and we can treat it as bugfix). We've already done the same in #35378 (and It was me who proposed skipping the flag and treating it as bugfix :D ... Now I remember . |
So if we had it two times already would be even a good "quality excercise" to double check that all CLI commands exit with a proper error code in case of error :-D |
Yeah I tried to think more on it to keep backwards-compat but I think we are ok -- will remove the param and make it the default behavior |
**Motivation**:
Currently, when using `airflow dags test`, there is no easy way to know programmatically if a DagRun fails since the state is not stored in DB. The way to do know relies on log lines as below:
```bash
state=$(airflow dags test exception_dag | grep "DagRun Finished" | awk -F, '{for(i=1;i<=NF;i++) if ($i ~ / state=/) print $i}' | awk -F= '{print $2}') if [[ $state == "failed" ]]; then exit 1 else exit 0 fi
```
This PR adds `--fail-on-dagrun-failure` flag to `airflow dags test` command which will return an exit code of 1 if DagRun fails and makes it easy to integrate in CI for testing.
kaxil
force-pushed
the
raise-error-dag-test
branch
from
3df3eba
to
d76891e
Compare
|
@jscheffl Done |
kaxil
changed the title
DagRun fails when running dag testDagRun fails while running dag test
|
The failed test is unrelated to the PR and succeeds locally: |
**Motivation**:
Currently, when using `airflow dags test`, there is no easy way to know programmatically if a DagRun fails since the state is not stored in DB. The way to do know relies on log lines as below:
```bash
state=$(airflow dags test exception_dag | grep "DagRun Finished" | awk -F, '{for(i=1;i<=NF;i++) if ($i ~ / state=/) print $i}' | awk -F= '{print $2}') if [[ $state == "failed" ]]; then exit 1 else exit 0 fi
```
This PR adds will return an exit code 1 when `airflow dags test` command if DagRun fails and makes it easy to integrate in CI for testing.
(cherry picked from commit 383ad31)
…36517) **Motivation**: Currently, when using `airflow dags test`, there is no easy way to know programmatically if a DagRun fails since the state is not stored in DB. The way to do know relies on log lines as below: ```bash state=$(airflow dags test exception_dag | grep "DagRun Finished" | awk -F, '{for(i=1;i<=NF;i++) if ($i ~ / state=/) print $i}' | awk -F= '{print $2}') if [[ $state == "failed" ]]; then exit 1 else exit 0 fi ``` This PR adds will return an exit code 1 when `airflow dags test` command if DagRun fails and makes it easy to integrate in CI for testing.
Motivation:
Currently, when using
airflow dags test, there is no easy way to know programmatically if a DagRun fails since the state is not stored in DB. The way to do know relies on log lines as below:This PR will return an exit code 1 when
airflow dags testcommand if DagRun fails and makes it easy to integrate in CI for testing.