You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Although Astro SDK records data into temporary tables, it does not offer any tooling for users to clean temporary tables.
We should provide users with a separate Airflow Operators/tasks/decorators that:
Identifies temp tables created by tasks in the same Dag Run
Deletes temp tables for that dagrun
Allows users to selectively specific tables to clean up instead of defaulting all temp tables created in the same dag run.
Open Question:
Wait for the delete operation to be complete vs submitting deletion via async and marking task as complete?
Acceptance criteria
Have full coverage with tests
Have at least one example DAG
Document this feature, both using doc strings and in a higher-level in our guides (both REFERENCE.md and TUTORIAL.md including limitations
Time-boxed: 2-3 days
Although Astro SDK records data into temporary tables, it does not offer any tooling for users to clean temporary tables.
We should provide users with a separate Airflow Operators/tasks/decorators that:
Open Question:
Acceptance criteria
Related issues:
The text was updated successfully, but these errors were encountered: