Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle SystemExit raised in the task. #36986

Merged
merged 2 commits into from Jan 26, 2024

Conversation

avkirilishin
Copy link
Contributor

@avkirilishin avkirilishin commented Jan 23, 2024

Closes: #36809

Currently, we don't handle system exit events in tasks (e.g. sys.exit()). Due to this, tasks remain in the running state, which can lead to undesired behavior in cases like #36809.

* Exiting with a zero or None code signifies success, and the task does not return any value.
* Exiting with other codes signifies an error.
Copy link
Member

@potiuk potiuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. Would be good to see why the Zombie case is happening but it's not necessary for this PR to fix it of course.

@potiuk potiuk merged commit 574d90f into apache:main Jan 26, 2024
56 checks passed
amoghrajesh pushed a commit to amoghrajesh/airflow that referenced this pull request Jan 30, 2024
* Handle SystemExit raised in the task.

* Add handling of system exit in tasks:
* Exiting with a zero or None code signifies success, and the task does not return any value.
* Exiting with other codes signifies an error.
@ephraimbuddy ephraimbuddy added this to the Airflow 2.8.2 milestone Feb 19, 2024
@ephraimbuddy ephraimbuddy added the type:bug-fix Changelog: Bug Fixes label Feb 19, 2024
ephraimbuddy pushed a commit that referenced this pull request Feb 20, 2024
* Handle SystemExit raised in the task.

* Add handling of system exit in tasks:
* Exiting with a zero or None code signifies success, and the task does not return any value.
* Exiting with other codes signifies an error.

(cherry picked from commit 574d90f)
ephraimbuddy pushed a commit that referenced this pull request Feb 20, 2024
* Handle SystemExit raised in the task.

* Add handling of system exit in tasks:
* Exiting with a zero or None code signifies success, and the task does not return any value.
* Exiting with other codes signifies an error.

(cherry picked from commit 574d90f)
ephraimbuddy pushed a commit that referenced this pull request Feb 22, 2024
* Handle SystemExit raised in the task.

* Add handling of system exit in tasks:
* Exiting with a zero or None code signifies success, and the task does not return any value.
* Exiting with other codes signifies an error.

(cherry picked from commit 574d90f)
abhishekbhakat pushed a commit to abhishekbhakat/my_airflow that referenced this pull request Mar 5, 2024
* Handle SystemExit raised in the task.

* Add handling of system exit in tasks:
* Exiting with a zero or None code signifies success, and the task does not return any value.
* Exiting with other codes signifies an error.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:bug-fix Changelog: Bug Fixes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ShortCircuitOperator does not properly retry when python_callable exits unexpectedly
5 participants