Change last week to last 48 hours since weekly queries will be expensive.

[tirkarthi]

Last week was used during development as default and got changed to last 24 hours. The last week option is expensive in large environments and might take a long time to return the results. Replace it with "last 48 hours" which is a better option.

[pierrejeambrun]

Are you observing some slow requests on your big installation ? What is the bottleneck ?

[tirkarthi]

I didn't run Airflow 3 as part of a large setup. We noticed slowness for these kind of aggregation queries in the legacy cluster activity page that does same aggregation for task and dagrun states over a period. I thought similar would apply here.

[pierrejeambrun]

Hmmm it would be interesting to see what is actually causing the delay and optimize those queries. Here we are on the same order of magnitude (from 168 to 48, 3 times bigger install will cause the same delay, so this problem will happen again).

Maybe we should leave 1 week because that's what we want feature wise, and then observe what's limiting and optimize those ?

[jscheffl]

+1 to Pierre. 168h to 48h does not eliminate the root. Either the query needs to be optimized or maybe also an additional DB index is needed to speed-up.

[tirkarthi]

Thanks @pierrejeambrun and @jscheffl. Closing this for now and will see if there is noticeable delay reported during RC testing and after release.

[tirkarthi]

@vatsrahul1001 I remember some discussion on testing Airflow release in a large environment with more data. Perhaps this could be tested to ensure weekly query is fast enough in the home page.