-
Notifications
You must be signed in to change notification settings - Fork 16.5k
Open
Labels
area:APIAirflow's REST/HTTP APIAirflow's REST/HTTP APIarea:UIRelated to UI/UX. For Frontend Developers.Related to UI/UX. For Frontend Developers.kind:metaHigh-level information important to the communityHigh-level information important to the community
Milestone
Description
This is the meta issue for tracking performance optimization of the UI and API to improve airflow operating at high scale.
I - API
- Add gunicorn support for API server with rolling worker restarts #60940 (memory optimization and worker rotation)
- API - Improve historical_metrics_data endpoint #62021
- API - Improve public get_dag_runs endpoint #62025
- API - Improve public get_task_instances endpoint #62027
- Mention http2 in the API/UI performance documentation #62032
- Optimize /ui/dags endpoint serialization #61483
- requires_access_dag checks made per grid ti summary API call rendering the dag page are redundant #61485 Cache dag access and teams check to be reused in grid ti_summary API calls #61623
II - UI
- Add virtualization to grid view #60241
- Optimize grid node merging with ID index #61656
- UI - Improve refresh rate strategy of the Grid view #62033
- Replace node-sql-parser with sqlparser-ts #61111
Attached is a zip file with some dummy dags that will generate a high number of tasks and runs in your local development setup. This is useful to 'simulate' a high scale installation with millions of TIs and thousands of dag runs, to help you work on the linked issues or identify other bottlenecks.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
area:APIAirflow's REST/HTTP APIAirflow's REST/HTTP APIarea:UIRelated to UI/UX. For Frontend Developers.Related to UI/UX. For Frontend Developers.kind:metaHigh-level information important to the communityHigh-level information important to the community