-
Notifications
You must be signed in to change notification settings - Fork 14.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Return nonzero exit codes on pool import errors. #12095
Conversation
The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*. |
The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*. |
c7cbd83
to
c676520
Compare
The PR should be OK to be merged with just subset of tests as it does not modify Core of Airflow. The committers might merge it or can add a label 'full tests needed' and re-run it to run all tests if they see it is needed! |
airflow/cli/commands/pool_command.py
Outdated
pools.append(api_client.create_pool(name=k, slots=v["slots"], description=v["description"])) | ||
else: | ||
failed.append(k) | ||
print("{} of {} pool(s) successfully updated.".format(len(pools), len(pools_json))) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we use f-strings?
airflow/cli/commands/pool_command.py
Outdated
else: | ||
failed.append(k) | ||
print("{} of {} pool(s) successfully updated.".format(len(pools), len(pools_json))) | ||
return pools, failed # pylint: disable=lost-exception |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
return pools, failed # pylint: disable=lost-exception | |
return pools, failed |
Should we be able to remove it?
c676520
to
895ddee
Compare
Thanks for reviewing @turbaszek, I updated the branch. |
895ddee
to
70ba350
Compare
The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*. |
70ba350
to
ed03cb9
Compare
@kaxil @turbaszek is it all right that some "quarantined" tests are failing? I don't think they're related to these changes. |
@jmcarp Some of the quarantined tests started to fail recently and we have an on-going effort to fix them before 2.0 is out. See the automated issue here: #10118 and
Quarantine
BTW. If you would like to help with that - feel free to pick any of those issues :). |
ed03cb9
to
9ac753e
Compare
The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*. |
9ac753e
to
e7cc6f9
Compare
Not sure what changed, but I rebased on master, and tests are passing now. Is this ok to merge? |
airflow/cli/commands/pool_command.py
Outdated
print("Failed to update pool(s): {}".format(", ".join(failed))) | ||
sys.exit(1) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
print("Failed to update pool(s): {}".format(", ".join(failed))) | |
sys.exit(1) | |
exit("Failed to update pool(s): {}".format(", ".join(failed))) |
This way the error goes to stderr, not stdout
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see that we use a mixture of sys.exit
and raise SystemExit
across the cli module. Do we prefer one or the other?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I picked sys.exit
arbitrarily for now. We can pick one invocation and use it consistently throughout the module separately.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One small change please
The pool import command returns an exit code of zero in a few different error cases. This causes problems for scripts that invoke the command, since commands that actually failed will appear to have worked. This patch returns a nonzero code if the pool file doesn't exist, if the file isn't valid json, or if any of the pools in the file is invalid.
e7cc6f9
to
d485cfc
Compare
The PR should be OK to be merged with just subset of tests as it does not modify Core of Airflow. The committers might merge it or can add a label 'full tests needed' and re-run it to run all tests if they see it is needed! |
The pool import command returns an exit code of zero in a few different
error cases. This causes problems for scripts that invoke the command,
since commands that actually failed will appear to have worked. This
patch returns a nonzero code if the pool file doesn't exist, if the file
isn't valid json, or if any of the pools in the file is invalid.
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.