Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include triggerer health status in Airflow /health endpoint #31529

Conversation

pankajkoti
Copy link
Member

PR #27755 introduced sending triggerer health status in the /api/v1/health
endpoint and also updated relevant docs but we have the primary /health
endpoint too which is missing this information. The PR addresses this missing
status report for triggerer health in the /health endpoint. It also attempts to
deduplicate the code between those endpoints so that in future when we need
to make necessary changes in only one place and at the same time ensure that
change made in one endpoint is not missed for the other endpoint serving the
same purpose and thus ensuring consistency in the responses.

fixes: #31522


^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@boring-cyborg boring-cyborg bot added area:API Airflow's REST/HTTP API area:webserver Webserver related Issues labels May 24, 2023
@pankajkoti pankajkoti changed the title Return triggerer health status in Airflow /health endpoint Include triggerer health status in Airflow /health endpoint May 24, 2023
@pankajkoti pankajkoti force-pushed the expose-triggerer-status-on-airflow-health-endpoint branch from 2b49b61 to 392481a Compare May 24, 2023 21:29
@pankajkoti
Copy link
Member Author

API responses now:
Screenshot 2023-05-25 at 3 28 24 AM

Screenshot 2023-05-25 at 3 28 02 AM Screenshot 2023-05-25 at 3 27 47 AM

@pankajkoti pankajkoti force-pushed the expose-triggerer-status-on-airflow-health-endpoint branch from 392481a to 8bfe5de Compare May 24, 2023 22:00
Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sweet! I could use that also in my Cluster Activity page to display the triggerer health 🎉

Maybe we can add a small test case ? (most of the utils are tested in test_utils)

@pankajkoti pankajkoti force-pushed the expose-triggerer-status-on-airflow-health-endpoint branch from 3ff9a49 to 6b7e659 Compare May 25, 2023 20:46
Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor suggestion but LGTM.

UNHEALTHY = "unhealthy"


def get_airflow_health():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we could add a return type annotation. (TypedDict, or Dict at least ?)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to be descriptive by specifying the return type as dict[str, dict[str, str | None]], but it failed both locally (I thought my breeze might be acting weird) and also on CI with the below error, so changed it to dict[str, Any] which breeze accepted. So, hopefully CI will accept it too 🤞🏽 .

Screenshot 2023-05-27 at 12 04 31 AM

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, CI is green now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you missed a nested level of dict, but I think any is acceptable here too.

@pankajkoti
Copy link
Member Author

1 failing check in CI :( I think this is unrelated, but I will try another rebase to re-trigger CI.

Screenshot 2023-05-27 at 12 59 45 AM

PR apache#27755 introduced sending triggerer
health status in the `/api/v1/health` endpoint and also updated relevant
docs but we've the primary `/health` too which is missing this information.
The PR addresses this missing status report for triggerer health in the
`/health` endpoint. It also attempts to deduplicate the code between those
endpoints so that in future we need to make necessary changes in only one
place and at the same time ensure that change made in one endpoint is not
missed for the other endpoint serving the same purpose and thus ensuring
consistency in the responses.

fixes: apache#31522
@pankajkoti pankajkoti force-pushed the expose-triggerer-status-on-airflow-health-endpoint branch from ba47a63 to 22994d7 Compare May 26, 2023 19:30
@jedcunningham jedcunningham merged commit f048aba into apache:main May 26, 2023
42 checks passed
@jedcunningham jedcunningham deleted the expose-triggerer-status-on-airflow-health-endpoint branch May 26, 2023 20:22
@jedcunningham jedcunningham added this to the Airflow 2.6.2 milestone May 26, 2023
@jedcunningham jedcunningham added the type:bug-fix Changelog: Bug Fixes label May 26, 2023
eladkal pushed a commit that referenced this pull request Jun 8, 2023
PR #27755 introduced sending triggerer
health status in the `/api/v1/health` endpoint and also updated relevant
docs but we've the primary `/health` too which is missing this information.
The PR addresses this missing status report for triggerer health in the
`/health` endpoint. It also attempts to deduplicate the code between those
endpoints so that in future we need to make necessary changes in only one
place and at the same time ensure that change made in one endpoint is not
missed for the other endpoint serving the same purpose and thus ensuring
consistency in the responses.

fixes: #31522
(cherry picked from commit f048aba)
eladkal pushed a commit that referenced this pull request Jun 9, 2023
PR #27755 introduced sending triggerer
health status in the `/api/v1/health` endpoint and also updated relevant
docs but we've the primary `/health` too which is missing this information.
The PR addresses this missing status report for triggerer health in the
`/health` endpoint. It also attempts to deduplicate the code between those
endpoints so that in future we need to make necessary changes in only one
place and at the same time ensure that change made in one endpoint is not
missed for the other endpoint serving the same purpose and thus ensuring
consistency in the responses.

fixes: #31522
(cherry picked from commit f048aba)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:API Airflow's REST/HTTP API area:webserver Webserver related Issues type:bug-fix Changelog: Bug Fixes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

/health endpoint missed when adding triggerer health status reporting
4 participants