fix(metrics): emit queued_duration metric in execution API ti_run endpoint#63522
fix(metrics): emit queued_duration metric in execution API ti_run endpoint#63522YoannAbriel wants to merge 1 commit intoapache:mainfrom
Conversation
|
@YoannAbriel This PR has been converted to draft because it does not yet meet our Pull Request quality criteria. Issues found:
What to do next:
Converting a PR to draft is not a rejection — it is an invitation to bring the PR up to the project's standards so that maintainer review time is spent productively. If you have questions, feel free to ask on the Airflow Slack. |
…point The task.queued_duration metric was not being emitted when tasks transitioned from QUEUED to RUNNING via the execution API (used by the task SDK in Airflow 3). The metric was only emitted in the legacy TaskInstance._run_raw_task path. Add queued_dttm, end_date, and queue to the ti_run select query and emit the metric after the state update, matching the behavior in TaskInstance.emit_state_change_metric. Closes: apache#63503
32a76f0 to
c47731a
Compare
|
@YoannAbriel This PR has been converted to draft because it does not yet meet our Pull Request quality criteria. Issues found:
What to do next:
Converting a PR to draft is not a rejection — it is an invitation to bring the PR up to the project's standards so that maintainer review time is spent productively. There is no rush — take your time and work at your own pace. We appreciate your contribution and are happy to wait for updates. If you have questions, feel free to ask on the Airflow Slack. |
In Airflow 3, tasks transition from QUEUED to RUNNING via the execution API (
ti_runendpoint), not the legacyTaskInstance._run_raw_taskpath. Thetask.queued_durationmetric (dag.{dag_id}.{task_id}.queued_durationin legacy format) is only emitted in the old path viaemit_state_change_metric(), so it's never recorded for tasks started through the task SDK.This adds the metric emission to
ti_run()— selectsqueued_dttm,end_date, andqueuefrom the TI row and callsDualStatsManager.timing()after the state update. Same first-attempt guard as the original (skips whenend_dateis set).Closes: #63503
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 4, claude-opus-4-6) following the guidelines