Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Expose count of scheduled tasks in metrics #38899

Merged
merged 1 commit into from
Apr 22, 2024
Merged

Expose count of scheduled tasks in metrics #38899

merged 1 commit into from
Apr 22, 2024

Conversation

jonshea
Copy link
Contributor

@jonshea jonshea commented Apr 10, 2024

This pull request introduces a new metric that tracks the number of tasks in the SCHEDULED state. This can help with monitoring and debugging situations where tasks are stuck or otherwise accumulate in the SCHEDULED state.

@boring-cyborg boring-cyborg bot added area:Scheduler including HA (high availability) scheduler kind:documentation labels Apr 10, 2024
@jonshea jonshea force-pushed the expose-scheduled-task-count-in-metrics branch from 30725fe to 2dd8091 Compare April 10, 2024 15:17
@jonshea jonshea force-pushed the expose-scheduled-task-count-in-metrics branch from 2dd8091 to 1f2a538 Compare April 22, 2024 07:49
@jonshea
Expose count of scheduled tasks in metrics
3ba77c8 
This pull request introduces a new metric that tracks the number of
tasks in the SCHEDULED state. This can help with monitoring and
debugging situations where tasks are stuck or otherwise accumulate in
the `SCHEDULED` state.
@jonshea jonshea force-pushed the expose-scheduled-task-count-in-metrics branch from 1f2a538 to 3ba77c8 Compare April 22, 2024 07:55
@jonshea
Copy link
Contributor Author

jonshea commented Apr 22, 2024

I have rebased on top of main to fix merge conflicts in test_pool.py. The conflicts were just that the pattern for creating a test task instance had changed from TI(task=op1, execution_date=DEFAULT_DATE) to TI(task=op1, run_id=dr.run_id).

@potiuk potiuk merged commit 667ee1b into apache:main Apr 22, 2024
42 checks passed
@rawwar rawwar mentioned this pull request May 25, 2024
Closed
2 tasks
@utkarsharma2 utkarsharma2 added the type:improvement Changelog: Improvements label Jun 3, 2024
@utkarsharma2 utkarsharma2 added this to the Airflow 2.10.0 milestone Jun 4, 2024
@mattogburke mattogburke mentioned this pull request Aug 30, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@potiuk potiuk potiuk approved these changes

@dirrao dirrao dirrao approved these changes

@kaxil kaxil Awaiting requested review from kaxil

@XD-DENG XD-DENG Awaiting requested review from XD-DENG XD-DENG is a code owner

@ashb ashb Awaiting requested review from ashb ashb is a code owner

Assignees
No one assigned
Labels
area:Scheduler including HA (high availability) scheduler kind:documentation type:improvement Changelog: Improvements
Projects
None yet
Milestone
Airflow 2.10.0
Development

Successfully merging this pull request may close these issues.
4 participants
@jonshea @potiuk @dirrao @utkarsharma2