Skip to content

Fix airflow db clean hang on MySQL when delete fails#66296

Open
Vamsi-klu wants to merge 2 commits into
apache:mainfrom
Vamsi-klu:fix-db-clean-mysql-hang-66177
Open

Fix airflow db clean hang on MySQL when delete fails#66296
Vamsi-klu wants to merge 2 commits into
apache:mainfrom
Vamsi-klu:fix-db-clean-mysql-hang-66177

Conversation

@Vamsi-klu
Copy link
Copy Markdown
Contributor

Fixes a MySQL hang in airflow db clean --skip-archive when a delete fails on a foreign-key constraint.

On the failure path, _do_delete() now rolls back the session before dropping the temporary archive table, and the drop uses the session's active connection instead of checking out a new pooled connection. This prevents metadata-lock contention from leaving the cleanup command stuck indefinitely.

Also adds:

  • unit coverage for rollback-before-drop ordering
  • a MySQL backend regression test for the deadlock case
  • a Postgres backend regression test for the failure path
  • a newsfragment

Validation:

  • prek run --from-ref main --stage pre-commit
  • prek run --from-ref main --stage manual
  • targeted MySQL and Postgres regression tests in Breeze
  • full airflow-core/tests/unit/utils/test_db_cleanup.py in Breeze (57 passed, 1 skipped)
  • breeze ci selective-check --commit-ref HEAD indicates the shared test utility change expands the CI matrix beyond what was practical to run locally

closes: #66177


Was generative AI tooling used to co-author this PR?
  • Yes — Codex

Generated-by: Codex following the guidelines

@csrinivas2130
Copy link
Copy Markdown

csrinivas2130 commented May 5, 2026

@Vamsi-klu can you close your PR, I have made the fix 4 days ago and waiting for the #PR-66249 approval for merge and close this issue.

@Vamsi-klu Vamsi-klu closed this May 10, 2026
@Vamsi-klu Vamsi-klu deleted the fix-db-clean-mysql-hang-66177 branch May 10, 2026 01:00
@Vamsi-klu Vamsi-klu reopened this May 13, 2026
@Vamsi-klu Vamsi-klu force-pushed the fix-db-clean-mysql-hang-66177 branch from 9967327 to 6e2e0bf Compare May 13, 2026 04:47
Vamsi-klu added 2 commits May 21, 2026 08:45
The check-newsfragment-pr-number CI job requires the newsfragment
filename to match the PR number, not the linked issue number.
@Vamsi-klu Vamsi-klu force-pushed the fix-db-clean-mysql-hang-66177 branch from 6e2e0bf to 7dec317 Compare May 21, 2026 15:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

airflow db clean --tables dag_version --skip-archive hangs forever (MDL deadlock) when an old dag_version is FK-pinned by task_instance

2 participants