New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[AIRFLOW-6867] Fix bug in how we call kill_zombies #7534
Conversation
The refactor in apache#7488 introduced a scheduler-breaking bug.
@@ -877,7 +877,7 @@ def process_file(self, file_path, zombies, pickle_dags=False, session=None): | |||
except Exception: | |||
self.log.exception("Error logging import errors!") | |||
try: | |||
self.kill_zombies(zombies) | |||
self.kill_zombies(dagbag, zombies) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Surprised that our static checkers didn't catch this! Required arg was not passed !
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah quite!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe pylint would have caught it but it is currently in pylint_todo, so might have missed:
airflow/scripts/ci/pylint_todo.txt
Line 11 in 6eaa7e3
./airflow/jobs/scheduler_job.py |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indeed .
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tomek has the PR #7484 in progress for that.
Duplicate of #7531 |
The refactor in #7488 introduced a bug.
I am not happy about merging this without a unit test to stop it breaking again, but given how much process_file does it's hard to test. Thoughts?
Issue link: AIRFLOW-6867
Make sure to mark the boxes below before creating PR: [x]
[AIRFLOW-NNNN]
. AIRFLOW-NNNN = JIRA ID** For document-only changes commit message can start with
[AIRFLOW-XXXX]
.In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.
Read the Pull Request Guidelines for more information.