Fix N+1 query pattern in bulk pool delete endpoint

[ColtenOuO]

Summary

Eliminates an N+1 query in the bulk pool delete endpoint by reusing the ORM objects already loaded by BulkPoolService.categorize_pools() instead of re-fetching each pool one at a time inside the delete loop.

Why

categorize_pools() already loads every requested pool with a single batched SELECT ... WHERE pool IN (...) and returns them as existing_pools_dict: dict[str, Pool]. handle_bulk_delete() was discarding that dict (_, matched, not_found = ...) and then re-querying each pool individually inside the loop:

for pool_name in delete_pool_names:
    existing_pool = self.session.scalar(select(Pool).where(Pool.pool == pool_name).limit(1))
    ...

For a request that deletes N pools, this produced 2N + 1 round-trips to the database (1 batched SELECT in categorize_pools + N redundant per-pool SELECTs + N DELETEs). The redundant per-pool SELECTs return objects we already have in memory.

Change 1

Behavior is preserved exactly: existing_pools_dict.get(pool_name) returns the same Pool instance the per-pool SELECT would have returned (it's the same object loaded by categorize_pools in the same session), and is None when the pool doesn't exist — matching the original session.scalar(...).limit(1) semantics.

Change 2

Added a unit test that asserts the expected query count.

Impact

Bulk size	Before (queries)	After (queries)	Saved
10 pools	21	11	-48%
100 pools	201	101	-50%

Unit Test Results

To ensure this optimization doesn't introduce any regressions, I've verified the changes with the existing test suite. All tests passed.

Command:
uv run --project airflow-core pytest airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_pools.py::TestBulkPools -x

Was generative AI tooling used to co-author this PR?

Yes — (Claude Sonnet 4.6, for test_bulk_delete_does_not_re_query_each_pool in airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_pools.py‎)

[SameerMesiah97]

Solid performance improvement.

[ColtenOuO]

I’ve noted that handle_bulk_update and update_orm_from_pydantic in pools.py likely suffer from similar N+1 query issues. I've intentionally kept this PR focused to verify if this optimization approach aligns with the project's expectations.

Once this logic is confirmed and merged, I will follow up with another PR to address the remaining functions. Thanks again to all the reviewers!

[jason810496]

LGTM overall, thanks for the improvement.
It would be nice to add or edit corresponding test case to guard the query count.

FYI, the assert_queries_count is the current convention to check the expected query count.

airflow/airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_dags.py

Lines 428 to 441 in 7336ed4

	def test_get_dags(self, test_client, query_params, expected_total_entries, expected_ids, session):
	# Only create asset test data for asset-related tests to avoid affecting other tests
	if any(param in query_params for param in ["has_asset_schedule", "asset_dependency"]):
	self._create_asset_test_data(session)
	with assert_queries_count(4):
	response = test_client.get("/dags", params=query_params)
	assert response.status_code == 200
	body = response.json()
	assert body["total_entries"] == expected_total_entries
	actual_ids = [dag["dag_id"] for dag in body["dags"]]
	assert actual_ids == expected_ids

[ColtenOuO]

Thanks for the suggestion @jason810496 — I've added a test that uses assert_queries_count per the convention you linked.

Additionally, while checking this unit test, I realized that using round-trip times provides a more accurate evaluation of the impact. I have updated the table below to reflect this:

Bulk size	Before	After	Saved
10 pools	14	4	−71%
100 pools	104	4	−96%

If you have any feedback or suggestions for this PR, feel free to let me know! Thanks to all the reviewers for your support!

[henry3260]

Thanks for your effort!

[henry3260]

Thanks! I think we are almost there

[jason810496]

LGTM overall, one final nit before merge. Thanks.

[jason810496]

Would it be better to simplify as accepting single pool_count? Since the expected count should be the same.

[ColtenOuO]

Thanks for the feedback @jason810496!

Refactored the code based on your suggestion. I initially over-engineered it to avoid the hardcoded query count.

Feel free to let me know if there’s anything else or if I misunderstood anything.

[ColtenOuO]

Fixed ruff formatting to pass CI (force-pushed to the latest commit).

[potiuk]

@ColtenOuO — There are 4 unresolved review threads on this PR from @jason810496. Could you either push a fix or reply in each thread explaining why the feedback doesn't apply? Once you believe the feedback is addressed, mark the thread as resolved so the reviewer isn't re-pinged needlessly. Thanks!

---

Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

[jason810496]

Thanks for the fix.

[github-actions]

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-2-test