Fix DAG processor OOM || Avoid loading all TaskInstances when checking DagVersion in write_dag

[vatsrahul1001]

During DAG serialization, write_dag uses joinedload to check if any task
instances exist for a DAG version. This loads all task instances into memory
just to answer a boolean question.

For long-running deployments with many DAG runs, this can cause high memory
usage during serialization.

This PR optimizes the check by using an EXISTS query instead, which has
constant memory usage regardless of how many task instances exist.

• Yes (please specify the tool below)

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

[jedcunningham]

I've been profiling this change locally. Here are the results with 600k TIs for the same dag version.

Before:

After:

[Copilot]

Pull request overview

This PR optimizes the DAG serialization process to prevent out-of-memory (OOM) issues in the DAG processor. The change replaces an inefficient joinedload operation that loads all TaskInstances into memory with a lightweight EXISTS query that uses constant memory, regardless of the number of task instances.

Changes:

• Replaced joinedload(DagVersion.task_instances) with an exists() query to check for task instance existence
• Added explicit TaskInstance import to support the new query pattern
• Removed unused joinedload import from sqlalchemy.orm

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

Backport failed to create: v3-1-test. View the failure log Run details

Status	Branch	Result
❌	v3-1-test

You can attempt to backport this manually by running:

cherry_picker 235595b v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

[vatsrahul1001]

Auto backport failed.

Manual backport to v3-1-test #60962