You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
While full db cleanup is useful to run in rare occasions, deleting data regarding to specific dag ids provides modular maintenance method to clean specific "intensive" dags and provide clear context and responsivness in the web ui and interfaces in cluster.
an initial idea was gathered to add optional dag_id key in _TableConfig at db_cleanup.py and append condition in _build_query(after adding the argument of course). Those changes could be provided also in the helm chart as list of cleanup jobs.
would love to hear you inputs about the use case and the implementation relating to the current effort(e.g #24590).
Thanks.
Use case/motivation
cleanup of specific "intensive" dags.
modular and configurable context and metadata in the web server and the different interfaces.
Feel free to implement it. the #24590 is going to be removed - we had recently a discussion in #24591 that led us to rethink if and how we are going to implement click, so there is no impact from that on your work.
It would be a nice feature to add filter by dag-id
Description
While full db cleanup is useful to run in rare occasions, deleting data regarding to specific dag ids provides modular maintenance method to clean specific "intensive" dags and provide clear context and responsivness in the web ui and interfaces in cluster.
an initial idea was gathered to add optional
dag_id
key in_TableConfig
at db_cleanup.py and append condition in_build_query
(after adding the argument of course). Those changes could be provided also in the helm chart as list of cleanup jobs.would love to hear you inputs about the use case and the implementation relating to the current effort(e.g #24590).
Thanks.
Use case/motivation
Related issues
No response
Are you willing to submit a PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: