After deleting a dag from the dags directory, it is still displayed in the UI

[tatyana12345]

Apache Airflow version

Other Airflow 2 version (please specify below)

If "Other Airflow 2 version" selected, which one?

2.8.3

What happened?

Dag has been removed from the airflow dags directory /opt/airflow/dags. Dag is not displayed when running the airflow dags list command. But it is still available in the Airflow UI, and it is also available in the Airflow postgres database.

What you think should happen instead?

It is expected that dag will be automatically removed from the web interface after deleting dag from the dags directory.

How to reproduce

Remove dag from the /opt/airflow/dags directory and it will still be available in the web interface even after a few days

Operating System

Linux

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[tirkarthi]

Deleting the Airflow dag file doesn't delete the db entries. You might want to try below command to delete the related entries. Please also check the below discussion that has different steps in deleting the dag completely.

#21864

airflow dags delete -h 
Usage: airflow dags delete [-h] [-v] [-y] dag_id

Delete all DB records related to the specified DAG

Positional Arguments:
  dag_id         The id of the dag

Options:
  -h, --help     show this help message and exit
  -v, --verbose  Make logging output more verbose
  -y, --yes      Do not prompt to confirm. Use with care!

[tatyana12345]

Thanks for your reply! I understand that I can delete a dag via the CLI or even from the database, but it seems inconvenient to delete the same dag from two places.
Do you have any information on why deleting the Airflow DAG file does not delete the database entries? This worked in previous versions, but no longer works in the current one.

[tirkarthi]

I am not sure such feature was present. Deleting dags in dags folder stops dag processor from picking it up and after a certain point depending on configurations the value is deleted in serialized_dag table with dag deactivated. But the metadata linked to dag still had to be manually deleted. This could be inconvenient and confusing but that's how it worked in the past and now. Adding relevant part from the discussion.

This all means that if you want to actually delete a DAG and its all historical metadata, you need to do
it in three steps:
pause the DAG
delete the historical metadata from the database, via UI or API
delete the DAG file from the DAGS_FOLDER and wait until it becomes inactive

[falukelo]

I got this issue, I am try this suggestion, but the deleted DAGs keep reappearing.

[shahar1]

Please try @tirkarthi 's suggestion.
I'm converting the issue into a discussion, as it seems like an expected behavior.