Skip to content

prevent DAG callback exception from crashing scheduler#10096

Merged
potiuk merged 3 commits intoapache:masterfrom
houqp:dag_callback
Aug 6, 2020
Merged

prevent DAG callback exception from crashing scheduler#10096
potiuk merged 3 commits intoapache:masterfrom
houqp:dag_callback

Conversation

@houqp
Copy link
Copy Markdown
Member

@houqp houqp commented Aug 1, 2020

Also adds type annotation so we can catch signal type exceptions at build time.


^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

@houqp houqp requested review from ashb, potiuk and turbaszek and removed request for turbaszek August 1, 2020 19:40
Copy link
Copy Markdown
Member

@potiuk potiuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! But I think this is sort-of breaking change so we should likely make a note about it in the UPDATING.md

Copy link
Copy Markdown
Member

@turbaszek turbaszek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good but as Jarek mentioned we should add a not to UPDATING.md. I like the idea of Context type!

@houqp houqp force-pushed the dag_callback branch 3 times, most recently from 61b512e to 22ecb53 Compare August 3, 2020 06:20
@houqp houqp requested a review from potiuk August 3, 2020 06:20
@houqp
Copy link
Copy Markdown
Member Author

houqp commented Aug 3, 2020

Added entry in UPDATING.md as well as added a counter metric to help better monitor this issue.

@houqp houqp force-pushed the dag_callback branch 2 times, most recently from 42c587b to c782ef2 Compare August 4, 2020 04:27
@houqp
Copy link
Copy Markdown
Member Author

houqp commented Aug 4, 2020

@turbaszek @potiuk fixed all test errors, ready for another round of review.

@houqp houqp requested a review from mik-laj August 4, 2020 16:53
Comment thread airflow/models/dag.py
@houqp houqp requested a review from mik-laj August 6, 2020 00:48
@potiuk potiuk merged commit 1e36666 into apache:master Aug 6, 2020
@houqp houqp deleted the dag_callback branch August 6, 2020 23:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants