Skip to content

Add dag_version cleanup indexes for task_instance and dag_run#63407

Open
shaealh wants to merge 8 commits intoapache:mainfrom
shaealh:shaealh-60145-issue/60145-add-missing-indexes
Open

Add dag_version cleanup indexes for task_instance and dag_run#63407
shaealh wants to merge 8 commits intoapache:mainfrom
shaealh:shaealh-60145-issue/60145-add-missing-indexes

Conversation

@shaealh
Copy link
Contributor

@shaealh shaealh commented Mar 12, 2026


Closes #60145

Adds a migration to speed up airflow db clean -t dag_version by creating missing indexes on:

  • task_instance(dag_version_id) (idx_task_instance_dag_version_id)
  • dag_run(created_dag_version_id) (idx_dag_run_created_dag_version_id)

Downgrade removes both indexes (if_exists=True).

Tests :

  • ruff check airflow-core/src/airflow/migrations/versions/0108_3_2_0_add_dag_version_id_indexes_for_db_cleanup.py
  • pytest airflow-core/tests/unit/utils/test_db.py -k single_head_revision_in_migrations -q --with-db-init
  • pytest airflow-core/tests/unit/migrations/test_no_orm_refs_in_migration_scripts.py -q

Was generative AI tooling used to co-author this PR?

  • Yes (please specify the tool below)
    Codex & ChatGPT

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@shaealh shaealh requested review from XD-DENG and ashb as code owners March 12, 2026 08:00
@shaealh
Copy link
Contributor Author

shaealh commented Mar 14, 2026

Please review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:db-migrations PRs with DB migration

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Possible missing indices on the database tables task_instance and dag_run

1 participant