Skip to content

Fix scheduler MySQL task instance index hint#66785

Open
8silvergun wants to merge 1 commit into
apache:mainfrom
8silvergun:fix-scheduler-mysql-ti-state-hint
Open

Fix scheduler MySQL task instance index hint#66785
8silvergun wants to merge 1 commit into
apache:mainfrom
8silvergun:fix-scheduler-mysql-ti-state-hint

Conversation

@8silvergun
Copy link
Copy Markdown
Contributor

closes: #66763

Summary

SchedulerJobRunner._executable_task_instances_to_queued applied USE INDEX (ti_state) twice on MySQL: once on the
inner ranked query that filters by scheduled task instances, and once on the outer rejoin that looks up
task_instance by (dag_id, task_id, run_id, map_index).

The outer query is an exact identity lookup, so forcing ti_state can prevent MySQL from using
task_instance_composite_key. This PR removes only the outer hint and keeps the inner hint that was added for the
original MySQL scheduler optimization.

Regression Coverage

Added a regression test that compiles the scheduler selection query with the MySQL dialect and verifies USE INDEX (ti_state) appears only once.

Tests

  • uv run --python 3.12 --group dev pytest airflow-core/tests/unit/jobs/ -xvs
  • prek run ruff --from-ref main
  • prek run ruff-format --from-ref main

Context


Was generative AI tooling used to co-author this PR?
  • Yes — OpenAI Codex

Generated-by: OpenAI Codex following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-
docs/05_pull_requests.rst#gen-ai-assisted-contributions)

@8silvergun 8silvergun requested review from XD-DENG and ashb as code owners May 12, 2026 15:25
# Select only rows where row_number <= max_active_tasks.
query = (
select(TI)
.with_hint(TI, "USE INDEX (ti_state)", dialect_name="mysql")
Copy link
Copy Markdown
Contributor

@eladkal eladkal May 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @xBis7 can you review this change. It was added in #54103

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eladkal Thanks for pinging me! I'm going to run some tests.

Copy link
Copy Markdown
Contributor

@xBis7 xBis7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally it looks good. I need to run some benchmark tests just to be sure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

4 participants