Add Dag version diff tooling

[ephraimbuddy]

Why

Dag version history preserves multiple serialized states, but operators do not have a supported way to answer what changed between two versions. Provide a deterministic comparison that helps diagnose version growth without requiring users to download and manually inspect serialized Dag payloads.

Keep the result honest about Airflow's persistence boundary. Treat every comparison as an observed-state diagnostic of the data currently stored for each version, not as creation-time evidence. Report missing, malformed, or unsupported historical data as unavailable instead of treating it as an empty diff.

This implements the shared service, CLI, and public API. Keep the UI panel as a later follow-up.

What changes

• Introduce a shared, versioned diff service for stored Dag versions.
• Normalize supported serialized Dag schema versions before comparison, including client defaults and order-insensitive collections, and produce deterministic JSON Pointer-style paths.
• Classify changes by operation, category, and advisory impact; include SHA-256 digests by default and optional raw values when permitted.
• Bound each response with max_changes, report truncation, and degrade unsafe canonicalization or missing payloads to an explicit unavailable result.
• Add airflow dags versions diff with explicit version selection or --previous-to-latest, optional values and source, configurable limits, and table, plain, JSON, or YAML output.
• Add GET /api/v2/dags/{dag_id}/dagVersions/{base_version_number}/diff/{target_version_number} and the corresponding response models.
• Require Dag version access for structured changes. Gate optional raw values against each version's stored bundle and team, and additionally enforce co-located Dag source permissions before loading or returning source code.
• Report source fidelity as current stored code, redacted, or unavailable. Do not present mutable DagCode rows as immutable historical snapshots.
• Regenerate the OpenAPI specification, UI API clients, AirflowCTL datamodels, and API permission reference.

Compatibility and scope

• Keep the change additive and read-only; require no metadata database migration.
• Leave Dag version creation, serialized Dag hashing, scheduler behavior, Dag processor behavior, worker bundle selection, and clear/rerun/backfill semantics unchanged.
• Preserve existing Dag version API responses.
• Add no user-facing UI in this PR; the UI changes are generated API client updates only, so screenshots do not apply.

Testing

• breeze run pytest airflow-core/tests/unit/models/test_dag_version_diff.py airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_dag_versions.py::TestGetDagVersionDiff airflow-core/tests/unit/cli/commands/test_dag_command.py::TestDagVersionDiffCommand airflow-core/tests/unit/cli/test_cli_parser.py::TestCli::test_dag_version_diff_parser -xvs — 50 passed.
• .venv/bin/prek run mypy-airflow-core --all-files
• .venv/bin/prek run generate-openapi-spec --all-files
• Ruff format and lint checks for all changed Python files.
• Repository pre-commit and commit-message hooks, including UI generation/lint and AirflowCTL datamodel generation.

---

Was generative AI tooling used to co-author this PR?

• Yes — Codex (GPT-5)

Generated-by: Codex (GPT-5) following the guidelines

[ephraimbuddy]

I will break this up into 3 parts. Please don't review yet