The cleanup
operator allows you to clean up temporary tables(table
) created as part of your Astro SDK pipeline. The cleanup
function monitors the status of all the tasks within your DAG, and deletes any temporary tables(table
) at the end of the DAG run. Astronomer recommends adding a cleanup
operator to every DAG that uses the Astro SDK.
../../../../example_dags/example_amazon_s3_postgres.py
You can also specify the temporary tables you want to delete by providing a list of tables with the parameter tables_to_cleanup
. Only temporary tables provided in this list will be deleted; persistent tables will be ignored.