Add (updated_at, dag_id) index to task_instance for updated_at-range API queries by seanmuth · Pull Request #65602 · apache/airflow

seanmuth · 2026-04-21T14:35:44Z

Summary

Adds one migration on task_instance: a composite index on
(updated_at, dag_id) to support the public REST API endpoint
GET /dags/~/dagRuns/~/taskInstances when called with an updated_at
range filter.

Without this index, the query — which adds a dag_id IN (...) clause
via PermittedTIFilter — performs a full sequential scan on
task_instance, observed at ~39s avg latency in RDS Performance
Insights on a real deployment. updated_at is the leading column so
Postgres can bound the scan to the time window; dag_id is the
trailing column to narrow within that range without a heap fetch.

Production evidence

Live pg_stat_user_indexes over a multi-week window across two
production deployments running this index out-of-band shows it heavily
used:

Deployment	`idx_scan` on `ti_updated_at_dag_id`	Index size
A	2,408,516	150 MB
B	2,541,802	16 MB

Notes

CONCURRENTLY is intentionally omitted — Alembic batch mode runs
inside a transaction. Operators with large existing tables who want a
lock-free build can apply the index manually with
CREATE INDEX CONCURRENTLY before running airflow db migrate.
An earlier revision of this PR included two additional indexes
(ti_state_updated_at partial index for terminal states, and
ti_span_status partial index). Both were dropped after production
index-usage stats showed them receiving ~0 scans on real workloads —
the queries that motivated them either no longer exist upstream or
were issued by an external service that has since been refactored.

Test plan

airflow db migrate runs cleanly from a current main schema
airflow db downgrade reverses the migration correctly
EXPLAIN ANALYZE on a loaded DB shows an index scan on
ti_updated_at_dag_id for the updated_at-range
taskInstances endpoint after migration
Existing task instance API tests pass (no schema regressions)

Was generative AI tooling used to co-author this PR?

Yes — Claude Code (Opus 4.7)

Generated-by: Claude Code (Opus 4.7) following the guidelines

…nup queries Adds two new migration files (0112, 0113) targeting high-volume query patterns on the task_instance table: - idx_CRE_ti_state_updated_at: partial index on (state, updated_at) filtered to terminal states (success, failed) — speeds up queries that scan completed tasks by updated_at without touching in-flight rows. - idx_CRE_ti_span_status: partial index on (span_status) filtered to 'should_end' — narrows the index to only the small fraction of rows the OTel span-closing logic needs to find, avoiding full-table scans. Both indexes use postgresql_where/sqlite_where for partial semantics on supported backends. MySQL receives non-partial fallback indexes (same pattern as existing idx_dag_run_running_dags). CONCURRENTLY is not used in the migration itself since Alembic batch mode runs inside a transaction; operators needing lock-free builds on large tables can apply the indexes manually with CONCURRENTLY before upgrading. Migration chain: 0111 (9fabad868fdb) → 0112 (c4f5e6d7a8b9) → 0113 (b0c1d2e3f4a5)

Adds migration 0114 and the corresponding __table_args__ entry for a composite index on (updated_at, dag_id). Without this index, the GET /dags/~/dagRuns/~/taskInstances endpoint with an updated_at range filter (plus the dag_id IN (...) clause from PermittedTIFilter) causes a full sequential scan on task_instance, observed at ~39s avg latency in RDS Performance Insights. Putting updated_at first lets Postgres bound the scan to the time window; dag_id as the second column avoids heap fetches for the predicate evaluation within that range. A partial index (as in 0112) would not help here because this query carries no state filter and must cover task instances in all states. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Renames idx_CRE_ti_state_updated_at → ti_state_updated_at and idx_CRE_ti_span_status → ti_span_status to match the ti_* convention used by all other task_instance indexes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

3.2.0 is already released. Rename files and update airflow_version to 3.3.0 to match the current development version and the convention established by 0110/0111 on this branch. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

seanmuth · 2026-04-21T14:37:11Z

Should be able to get some real-world query plan before/after's today

seanmuth · 2026-04-21T16:28:44Z

0114 dag_id updated_at

(Note: the actual query produced here ends with a WHERE ... task_instance.dag_id IN ('list', 'of', 'dag_ids') but the resulting query is longer than the psql default 4096 char limit, so I've substituted ... IN (select dag_id from dag) to mimic the "worst-case" scenario where the query is being run over a list of 1000s of dags, which is the scenario we found this query from)

Before:

=> EXPLAIN ANALYZE SELECT task_instance.rendered_map_index, task_instance.task_display_name, task_instance.id, task_instance.task_id, task_instance.dag_id, task_instance.run_id, task_instance.map_index, task_instance.start_date, task_instance.end_date, task_instance.duration, task_instance.state, task_instance.try_number, task_instance.max_tries, task_instance.hostname, task_instance.unixname, task_instance.pool, task_instance.pool_slots, task_instance.queue, task_instance.priority_weight, task_instance.operator, task_instance.custom_operator_name, task_instance.queued_dttm, task_instance.scheduled_dttm, task_instance.queued_by_job_id, task_instance.last_heartbeat_at, task_instance.pid, task_instance.executor, task_instance.executor_config, task_instance.updated_at, task_instance.context_carrier, task_instance.span_status, task_instance.external_executor_id, task_instance.trigger_id, task_instance.trigger_timeout, task_instance.next_method, task_instance.next_kwargs, task_instance.dag_version_id, dag_bundle_1.name, dag_bundle_1.active, dag_bundle_1.version, dag_bundle_1.last_refreshed, dag_bundle_1.signed_url_template, dag_bundle_1.template_params, dag_version_1.id AS id_1, dag_version_1.version_number, dag_version_1.dag_id AS dag_id_1, dag_version_1.bundle_name, dag_version_1.bundle_version, dag_version_1.created_at, dag_version_1.last_updated, dag_run_1.state AS state_1, dag_1.dag_display_name, dag_1.deadline, dag_1.dag_id AS dag_id_2, dag_1.is_paused, dag_1.is_stale, dag_1.last_parsed_time, dag_1.last_parse_duration, dag_1.last_expired, dag_1.fileloc, dag_1.relative_fileloc, dag_1.bundle_name AS bundle_name_1, dag_1.bundle_version AS bundle_version_1, dag_1.owners, dag_1.description, dag_1.timetable_summary, dag_1.timetable_description, dag_1.asset_expression, dag_1.max_active_tasks, dag_1.max_active_runs, dag_1.max_consecutive_failed_dag_runs, dag_1.has_task_concurrency_limits, dag_1.has_import_errors, dag_1.next_dagrun, dag_1.next_dagrun_data_interval_start, dag_1.next_dagrun_data_interval_end, dag_1.next_dagrun_create_after, dag_run_1.id AS id_2, dag_run_1.dag_id AS dag_id_3, dag_run_1.queued_at, dag_run_1.logical_date, dag_run_1.start_date AS start_date_1, dag_run_1.end_date AS end_date_1, dag_run_1.run_id AS run_id_1, dag_run_1.creating_job_id, dag_run_1.run_type, dag_run_1.triggered_by, dag_run_1.triggering_user_name, dag_run_1.conf, dag_run_1.data_interval_start, dag_run_1.data_interval_end, dag_run_1.run_after, dag_run_1.last_scheduling_decision, dag_run_1.log_template_id, dag_run_1.updated_at AS updated_at_1, dag_run_1.clear_number, dag_run_1.backfill_id, dag_run_1.bundle_version AS bundle_version_2, dag_run_1.scheduled_by_job_id, dag_run_1.context_carrier AS context_carrier_1, dag_run_1.span_status AS span_status_1, dag_run_1.created_dag_version_id, task_instance_note_1.ti_id, task_instance_note_1.user_id, task_instance_note_1.content, task_instance_note_1.created_at AS created_at_1, task_instance_note_1.updated_at AS updated_at_2
FROM task_instance JOIN dag_run ON dag_run.dag_id = task_instance.dag_id AND dag_run.run_id = task_instance.run_id LEFT OUTER JOIN dag_version ON dag_version.id = task_instance.dag_version_id LEFT OUTER JOIN dag_version
AS dag_version_1 ON dag_version_1.id = task_instance.dag_version_id LEFT OUTER JOIN dag_bundle AS dag_bundle_1 ON dag_version_1.bundle_name = dag_bundle_1.name JOIN dag_run AS dag_run_1 ON dag_run_1.dag_id = task_instance.dag_id AND dag_run_1.run_id = task_instance.run_id LEFT OUTER JOIN dag AS dag_1 ON dag_run_1.dag_id = dag_1.dag_id LEFT OUTER JOIN task_instance_note AS task_instance_note_1 ON task_instance.id = task_instance_note_1.ti_id
WHERE task_instance.updated_at >= '2026-04-21T07:00:00+00:00'::timestamptz AND task_instance.updated_at < '2026-04-21T15:00:00+00:00'::timestamptz AND task_instance.dag_id IN (select dag_id from dag);
                                                                                              QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Gather  (cost=81632.46..923436.39 rows=68310 width=3169) (actual time=2525.796..35686.290 rows=58040 loops=1)
   Workers Planned: 2
   Workers Launched: 0
   ->  Hash Join  (cost=80632.46..915605.39 rows=28462 width=3169) (actual time=2524.265..35569.585 rows=58040 loops=1)
         Hash Cond: ((dag_run.dag_id)::text = (dag.dag_id)::text)
         ->  Hash Left Join  (cost=80261.97..915160.13 rows=28462 width=3198) (actual time=2522.938..35534.218 rows=58040 loops=1)
               Hash Cond: (task_instance.id = task_instance_note_1.ti_id)
               ->  Nested Loop Left Join  (cost=80260.84..915084.28 rows=28462 width=2376) (actual time=2522.915..35503.392 rows=58040 loops=1)
                     ->  Hash Left Join  (cost=80260.54..913816.48 rows=28462 width=1865) (actual time=2522.895..35434.210 rows=58040 loops=1)
                           Hash Cond: ((dag_version_1.bundle_name)::text = (dag_bundle_1.name)::text)
                           ->  Nested Loop Left Join  (cost=80255.48..913631.16 rows=28462 width=1573) (actual time=2522.871..35401.171 rows=58040 loops=1)
                                 ->  Nested Loop  (cost=80255.05..907034.29 rows=28462 width=1446) (actual time=2522.853..35339.836 rows=58040 loops=1)
                                       ->  Parallel Hash Join  (cost=80254.48..166321.31 rows=237683 width=425) (actual time=2068.636..3370.229 rows=1018592 loops=1)
                                             Hash Cond: (((dag_run.dag_id)::text = (dag_run_1.dag_id)::text) AND ((dag_run.run_id)::text = (dag_run_1.run_id)::text))
                                             ->  Parallel Seq Scan on dag_run  (cost=0.00..53982.79 rows=424579 width=68) (actual time=0.006..237.348 rows=1018592 loops=1)
                                             ->  Parallel Hash  (cost=53982.79..53982.79 rows=424579 width=357) (actual time=1453.041..1453.041 rows=1018592 loops=1)
                                                   Buckets: 32768  Batches: 64  Memory Usage: 5824kB
                                                   ->  Parallel Seq Scan on dag_run dag_run_1  (cost=0.00..53982.79 rows=424579 width=357) (actual time=0.005..442.481 rows=1018592 loops=1)
                                       ->  Memoize  (cost=0.57..3.22 rows=1 width=1060) (actual time=0.031..0.031 rows=0 loops=1018592)
                                             Cache Key: dag_run.dag_id, dag_run.run_id
                                             Cache Mode: logical
                                             Hits: 0  Misses: 1018592  Evictions: 965176  Overflows: 0  Memory Usage: 8193kB
                                             ->  Index Scan using ti_dag_run on task_instance  (cost=0.56..3.21 rows=1 width=1060) (actual time=0.029..0.030 rows=0 loops=1018592)
                                                   Index Cond: (((dag_id)::text = (dag_run.dag_id)::text) AND ((run_id)::text = (dag_run.run_id)::text))
                                                   Filter: ((updated_at >= '2026-04-21 07:00:00+00'::timestamp with time zone) AND (updated_at < '2026-04-21 15:00:00+00'::timestamp with time zone))
                                                   Rows Removed by Filter: 13
                                 ->  Memoize  (cost=0.43..0.72 rows=1 width=127) (actual time=0.001..0.001 rows=1 loops=58040)
                                       Cache Key: task_instance.dag_version_id
                                       Cache Mode: logical
                                       Hits: 57006  Misses: 1034  Evictions: 0  Overflows: 0  Memory Usage: 247kB
                                       ->  Index Scan using dag_version_pkey on dag_version dag_version_1  (cost=0.42..0.71 rows=1 width=127) (actual time=0.015..0.015 rows=1 loops=1034)
                                             Index Cond: (id = task_instance.dag_version_id)
                           ->  Hash  (cost=5.03..5.03 rows=3 width=292) (actual time=0.013..0.014 rows=3 loops=1)
                                 Buckets: 1024  Batches: 1  Memory Usage: 9kB
                                 ->  Seq Scan on dag_bundle dag_bundle_1  (cost=0.00..5.03 rows=3 width=292) (actual time=0.004..0.010 rows=3 loops=1)
                     ->  Memoize  (cost=0.29..0.31 rows=1 width=511) (actual time=0.000..0.000 rows=1 loops=58040)
                           Cache Key: dag_run_1.dag_id
                           Cache Mode: logical
                           Hits: 57131  Misses: 909  Evictions: 0  Overflows: 0  Memory Usage: 516kB
                           ->  Index Scan using dag_pkey on dag dag_1  (cost=0.28..0.30 rows=1 width=511) (actual time=0.008..0.008 rows=1 loops=909)
                                 Index Cond: ((dag_id)::text = (dag_run_1.dag_id)::text)
               ->  Hash  (cost=1.06..1.06 rows=6 width=822) (actual time=0.016..0.016 rows=6 loops=1)
                     Buckets: 1024  Batches: 1  Memory Usage: 9kB
                     ->  Seq Scan on task_instance_note task_instance_note_1  (cost=0.00..1.06 rows=6 width=822) (actual time=0.005..0.006 rows=6 loops=1)
         ->  Hash  (cost=309.47..309.47 rows=4881 width=30) (actual time=1.285..1.285 rows=2953 loops=1)
               Buckets: 8192  Batches: 1  Memory Usage: 244kB
               ->  Index Only Scan using dag_pkey on dag  (cost=0.28..309.47 rows=4881 width=30) (actual time=0.014..0.673 rows=2953 loops=1)
                     Heap Fetches: 552
 Planning Time: 6.166 ms
 Execution Time: 35692.160 ms

After:

                                                                                                      QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Nested Loop  (cost=1008.87..234132.43 rows=65663 width=3190) (actual time=1.395..616.512 rows=58040 loops=1)
   ->  Gather  (cost=1008.58..231946.95 rows=65663 width=3219) (actual time=1.385..537.908 rows=58040 loops=1)
         Workers Planned: 2
         Workers Launched: 2
         ->  Hash Left Join  (cost=8.58..224380.65 rows=27360 width=3219) (actual time=0.226..559.899 rows=19347 loops=3)
               Hash Cond: (task_instance.id = task_instance_note_1.ti_id)
               ->  Nested Loop Left Join  (cost=7.45..224307.70 rows=27360 width=2397) (actual time=0.202..550.350 rows=19347 loops=3)
                     ->  Nested Loop  (cost=7.16..223072.07 rows=27360 width=1886) (actual time=0.181..522.064 rows=19347 loops=3)
                           ->  Hash Left Join  (cost=6.61..199875.17 rows=27360 width=1568) (actual time=0.156..297.605 rows=19347 loops=3)
                                 Hash Cond: ((dag_version_1.bundle_name)::text = (dag_bundle_1.name)::text)
                                 ->  Nested Loop Left Join  (cost=1.54..199696.82 rows=27360 width=1276) (actual time=0.120..287.439 rows=19347 loops=3)
                                       ->  Nested Loop  (cost=1.11..193158.23 rows=27360 width=1149) (actual time=0.102..267.731 rows=19347 loops=3)
                                             ->  Parallel Index Scan using idx_cre_ti_updated_at_dag_id on task_instance  (cost=0.56..117343.78 rows=27360 width=1081) (actual time=0.030..18.357 rows=19347 loops=3)
                                                   Index Cond: ((updated_at >= '2026-04-21 07:00:00+00'::timestamp with time zone) AND (updated_at < '2026-04-21 15:00:00+00'::timestamp with time zone))
                                             ->  Index Only Scan using dag_run_dag_id_run_id_key on dag_run  (cost=0.55..2.77 rows=1 width=68) (actual time=0.012..0.012 rows=1 loops=58040)
                                                   Index Cond: ((dag_id = (task_instance.dag_id)::text) AND (run_id = (task_instance.run_id)::text))
                                                   Heap Fetches: 22022
                                       ->  Memoize  (cost=0.43..0.73 rows=1 width=127) (actual time=0.001..0.001 rows=1 loops=58040)
                                             Cache Key: task_instance.dag_version_id
                                             Cache Mode: logical
                                             Hits: 16810  Misses: 880  Evictions: 0  Overflows: 0  Memory Usage: 210kB
                                             Worker 0:  Hits: 19679  Misses: 910  Evictions: 0  Overflows: 0  Memory Usage: 217kB
                                             Worker 1:  Hits: 18876  Misses: 885  Evictions: 0  Overflows: 0  Memory Usage: 212kB
                                             ->  Index Scan using dag_version_pkey on dag_version dag_version_1  (cost=0.42..0.72 rows=1 width=127) (actual time=0.004..0.004 rows=1 loops=2675)
                                                   Index Cond: (id = task_instance.dag_version_id)
                                 ->  Hash  (cost=5.03..5.03 rows=3 width=292) (actual time=0.020..0.020 rows=3 loops=3)
                                       Buckets: 1024  Batches: 1  Memory Usage: 9kB
                                       ->  Seq Scan on dag_bundle dag_bundle_1  (cost=0.00..5.03 rows=3 width=292) (actual time=0.008..0.013 rows=3 loops=3)
                           ->  Index Scan using dag_run_dag_id_run_id_key on dag_run dag_run_1  (cost=0.55..0.85 rows=1 width=357) (actual time=0.011..0.011 rows=1 loops=58040)
                                 Index Cond: (((dag_id)::text = (dag_run.dag_id)::text) AND ((run_id)::text = (dag_run.run_id)::text))
                     ->  Memoize  (cost=0.29..0.31 rows=1 width=511) (actual time=0.001..0.001 rows=1 loops=58040)
                           Cache Key: dag_run_1.dag_id
                           Cache Mode: logical
                           Hits: 16896  Misses: 794  Evictions: 0  Overflows: 0  Memory Usage: 454kB
                           Worker 0:  Hits: 19772  Misses: 817  Evictions: 0  Overflows: 0  Memory Usage: 465kB
                           Worker 1:  Hits: 18957  Misses: 804  Evictions: 0  Overflows: 0  Memory Usage: 459kB
                           ->  Index Scan using dag_pkey on dag dag_1  (cost=0.28..0.30 rows=1 width=511) (actual time=0.006..0.006 rows=1 loops=2415)
                                 Index Cond: ((dag_id)::text = (dag_run_1.dag_id)::text)
               ->  Hash  (cost=1.06..1.06 rows=6 width=822) (actual time=0.012..0.013 rows=6 loops=3)
                     Buckets: 1024  Batches: 1  Memory Usage: 9kB
                     ->  Seq Scan on task_instance_note task_instance_note_1  (cost=0.00..1.06 rows=6 width=822) (actual time=0.008..0.008 rows=6 loops=3)
   ->  Memoize  (cost=0.29..0.31 rows=1 width=30) (actual time=0.000..0.000 rows=1 loops=58040)
         Cache Key: dag_run.dag_id
         Cache Mode: logical
         Hits: 57131  Misses: 909  Evictions: 0  Overflows: 0  Memory Usage: 141kB
         ->  Index Only Scan using dag_pkey on dag  (cost=0.28..0.30 rows=1 width=30) (actual time=0.004..0.004 rows=1 loops=909)
               Index Cond: (dag_id = (dag_run.dag_id)::text)
               Heap Fetches: 595
 Planning Time: 7.390 ms
 Execution Time: 619.691 ms

potiuk · 2026-04-23T01:09:13Z

@seanmuth Converting to draft — this PR doesn't yet meet our Pull Request quality criteria.

❌ Failing CI: Sqlite tests: core / DB-core:Sqlite:3.10:Core...Serialization, Sqlite tests: core / DB-core:Sqlite:3.10:API...CLI, Postgres tests: core / DB-core:Postgres:14:3.10:API...CLI (+4 more). Please investigate and fix.

See the linked criteria for how to fix each item, then mark the PR "Ready for review". This is not a rejection — just an invitation to bring the PR up to standard. No rush.

Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

seanmuth · 2026-04-23T18:07:56Z

oops didn't mean to convert back from draft.

will work on getting those tests to pass today/tmr

ashb · 2026-04-23T20:12:52Z

@seanmuth Is the span_status still used on 3.2.0+ I know Standish did some work on overhauling Otel Traces.

seanmuth · 2026-04-23T21:06:16Z

@seanmuth Is the span_status still used on 3.2.0+ I know Standish did some work on overhauling Otel Traces.

will dig into that codepath

seanmuth · 2026-04-23T21:12:45Z

ah yep, removed as of 3.2 in #63452 , will pull that index/migration out of the PR shortly

Mirror the ti_state_updated_at partial index in __table_args__ so the SQLAlchemy model matches the migrated schema. Drop the unused ti_span_status migration (the OTEL span_status code was removed in 3.2) and reorder the remaining migrations.

potiuk · 2026-05-05T17:48:20Z

@seanmuth — There are 1 unresolved review thread(s) on this PR from @ashb. Could you either push a fix or reply in each thread explaining why the feedback doesn't apply? Once you believe the feedback is addressed, mark the thread as resolved so the reviewer isn't re-pinged needlessly. Thanks!

Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

Two production deployments observed over a multi-week window show idx_cre_ti_state_updated_at receiving only 2 and 28 index scans respectively, against tables where ti_dag_run / ti_state see tens of millions of scans. The terminal-state polling query that motivated the index was an external service that has since been refactored away — no upstream Airflow callsite issues a query matching the partial predicate (state IN ('success','failed') AND updated_at <op> ...). Carrying the index costs write amplification on every task transition into a terminal state plus tens of MB per deployment, with no corresponding read benefit. Drop the migration and the model index. The composite (updated_at, dag_id) index is retained — production stats show 2.4M and 2.5M scans on the same deployments, confirming it serves the public taskInstances updated_at-range endpoint as intended.

potiuk · 2026-05-18T09:19:59Z

@seanmuth — There is 1 unresolved review thread on this PR from @ashb, and you have engaged with each one (post-review commits and/or in-thread replies). Could you confirm whether you believe the feedback is fully addressed and the PR is ready for maintainer review confirmation?

If yes, reply here (a short "yes / ready" is fine) and an Apache Airflow maintainer will pick the PR up from the review queue on the next sweep.

If you are still working on a thread, please reply with what is outstanding so the threads stay unresolved on purpose.

Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

seanmuth · 2026-05-20T15:07:29Z

Sorry @potiuk forgot to mark that convo as resolved. Should be good to go now with just one index addition here.

potiuk · 2026-05-20T15:54:19Z

I think this is a migration that should be very carefully benchmarked. How long it takes to run it on a huge database ? Will it increase the size of the database ? Will it increase time to create entries ? I am not a specialist in this part -there are other people - bit at the very least evidences of migration checks on real data should be shown.

This is also unclear what should drive the decision on running it And how users should be warned they have to update the index with/without airflow running concurrently or not . I think any kind of migration of our DB should have evidence of testing it heavily on various types of deployments.

Also cc: @vatsrahul1001 and @ephraimbuddy who had to deal with lots of aftermath of not well tested migrations.

seanmuth and others added 4 commits April 20, 2026 12:58

seanmuth requested review from XD-DENG, ashb and ephraimbuddy as code owners April 21, 2026 14:35

boring-cyborg Bot added area:db-migrations PRs with DB migration kind:documentation labels Apr 21, 2026

This comment was marked as outdated.

Sign in to view

potiuk marked this pull request as draft April 23, 2026 01:09

seanmuth marked this pull request as ready for review April 23, 2026 18:04

seanmuth marked this pull request as draft April 23, 2026 18:04

seanmuth added 2 commits April 23, 2026 13:09

Merge remote-tracking branch 'apache' into add-helpful-af3-indexes

890cb49

add indexes to model defs

6a24d0b

seanmuth added 2 commits April 28, 2026 11:49

Merge remote-tracking branch 'apache/main' into add-helpful-af3-indexes

9815c89

seanmuth marked this pull request as ready for review April 28, 2026 17:58

ashb reviewed Apr 28, 2026

View reviewed changes

Comment thread ...low-core/src/airflow/migrations/versions/0112_3_3_0_add_ti_state_updated_at_partial_index.py Outdated

seanmuth changed the title ~~Add indexes on task_instance for span cleanup, terminal state, and updated_at polling queries~~ Add (updated_at, dag_id) index to task_instance for updated_at-range API queries May 12, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add (updated_at, dag_id) index to task_instance for updated_at-range API queries#65602

Add (updated_at, dag_id) index to task_instance for updated_at-range API queries#65602
seanmuth wants to merge 9 commits into
apache:mainfrom
seanmuth:add-helpful-af3-indexes

seanmuth commented Apr 21, 2026 •

edited

Loading

Uh oh!

seanmuth commented Apr 21, 2026

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

seanmuth commented Apr 21, 2026

Uh oh!

potiuk commented Apr 23, 2026 •

edited

Loading

Uh oh!

seanmuth commented Apr 23, 2026

Uh oh!

ashb commented Apr 23, 2026

Uh oh!

seanmuth commented Apr 23, 2026

Uh oh!

seanmuth commented Apr 23, 2026

Uh oh!

Uh oh!

potiuk commented May 5, 2026

Uh oh!

potiuk commented May 18, 2026

Uh oh!

seanmuth commented May 20, 2026

Uh oh!

potiuk commented May 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

seanmuth commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Production evidence

Notes

Test plan

Was generative AI tooling used to co-author this PR?

Uh oh!

seanmuth commented Apr 21, 2026

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

seanmuth commented Apr 21, 2026

0114 dag_id updated_at

Uh oh!

potiuk commented Apr 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

seanmuth commented Apr 23, 2026

Uh oh!

ashb commented Apr 23, 2026

Uh oh!

seanmuth commented Apr 23, 2026

Uh oh!

seanmuth commented Apr 23, 2026

Uh oh!

Uh oh!

potiuk commented May 5, 2026

Uh oh!

potiuk commented May 18, 2026

Uh oh!

seanmuth commented May 20, 2026

Uh oh!

potiuk commented May 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

seanmuth commented Apr 21, 2026 •

edited

Loading

potiuk commented Apr 23, 2026 •

edited

Loading