Not able to delete the DAG from UI -

[Gaurang033]

Apache Airflow version: 1.10.10

Kubernetes version (if you are using kubernetes) (use kubectl version):

Environment: prod

• Cloud provider or hardware configuration: aws

• OS (e.g. from /etc/os-release):
  NAME="Amazon Linux AMI"
  VERSION="2018.03"
  ID="amzn"
  ID_LIKE="rhel fedora"
  VERSION_ID="2018.03"
  PRETTY_NAME="Amazon Linux AMI 2018.03"
  ANSI_COLOR="0;33"
  CPE_NAME="cpe:/o:amazon:linux:2018.03:ga"
  HOME_URL="http://aws.amazon.com/amazon-linux-ami/"

• Kernel (e.g. uname -a): Linux ip-10-160-1-11 4.14.154-99.181.amzn1.x86_64 Improving the search functionality in the graph view #1 SMP Sat Nov 16 01:38:34 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

• Install tools:

• Others:

What happened:
While deleting example dags from UI, I am getting following error. I think the foreign key contstain for dag_tag table has not been handled properly.

If I modify the dag and remove tags, I am successfully able to remove the DAG.

What you expected to happen:
Delete dag without any error.

Traceback (most recent call last):
File "/usr/local/lib64/python2.7/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib64/python2.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib64/python2.7/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib64/python2.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib64/python2.7/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functionsrule.endpoint
File "/usr/local/lib/python2.7/site-packages/airflow/www_rbac/decorators.py", line 121, in wrapper
return f(self, *args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/flask_appbuilder/security/decorators.py", line 101, in wraps
return f(self, *args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/airflow/www_rbac/decorators.py", line 56, in wrapper
return f(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/airflow/www_rbac/views.py", line 972, in delete
delete_dag.delete_dag(dag_id)
File "/usr/local/lib/python2.7/site-packages/airflow/utils/db.py", line 74, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/airflow/api/common/experimental/delete_dag.py", line 61, in delete_dag
count += session.query(model).filter(cond).delete(synchronize_session='fetch')
File "/usr/local/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 3789, in delete
delete_op.exec_()
File "/usr/local/lib64/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 1697, in exec_
self._do_exec()
File "/usr/local/lib64/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 1928, in _do_exec
self._execute_stmt(delete_stmt)
File "/usr/local/lib64/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 1702, in _execute_stmt
self.result = self.query._execute_crud(stmt, self.mapper)
File "/usr/local/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 3436, in _execute_crud
return conn.execute(stmt, self._params)
File "/usr/local/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 984, in execute
return meth(self, multiparams, params)
File "/usr/local/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 293, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/local/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1103, in _execute_clauseelement
distilled_params,
File "/usr/local/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1288, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1482, in handle_dbapi_exception
sqlalchemy_exception, with_traceback=exc_info[2], from=e
File "/usr/local/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 588, in do_execute
cursor.execute(statement, parameters)
IntegrityError: (psycopg2.errors.ForeignKeyViolation) update or delete on table "dag" violates foreign key constraint "dag_tag_dag_id_fkey" on table "dag_tag"
DETAIL: Key (dag_id)=(example_pig_operator) is still referenced from table "dag_tag".

[SQL: DELETE FROM dag WHERE dag.dag_id = %(dag_id_1)s OR dag.dag_id LIKE %(dag_id_2)s]
[parameters: {'dag_id_1': u'example_pig_operator', 'dag_id_2': u'example_pig_operator.%'}]
(Background on this error at: http://sqlalche.me/e/gkpj)

How to reproduce it:

• Login to UI

• click on delete button for any DAG

Anything else we need to know: N/A

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[eladkal]

I'm unable to reproduce this on 2.0.0b3

[Gaurang033]

@eladkal does you're dag has any tags associated with it? This happens if DAG has any tag, seems like some foreign key constrain while deleting dag from database.

[eladkal]

@Gaurang033 yes. Can you please recheck with Airflow 2.0.0b3 ?
It could be issue with migration script if you upgraded from older version.
We'll need exact reproduce steps that recreates the issue.

[Gaurang033]

@eladkal we are using it in production, 2.0.0 is a long way to go.

[eladkal]

@Gaurang033 Airflow 1.10 is EOL
Closing as its working fine on Master (also #15832 is fixed)

[e-compagno]

with airflow 2.1.3 I get the error

Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
    self.dialect.do_execute(
  File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.ForeignKeyViolation: update or delete on table "dag" violates foreign key constraint "dag_tag_dag_id_fkey" on table "dag_tag"
DETAIL:  Key (dag_id)=(trial_test_dag) is still referenced from table "dag_tag".

Apparently trial_test_dag appears in tables dag and dag_tag if the db even if the dag code has been removed.

[gurudevkumar51]

I am also facing same issue when trying to clean DagModel from airflow.

[potiuk]

@gurudevkumar51 then either UPGRADE to fix the problem (you can look at the history and see linked PRs and see when they were fixed) or UPGRADE to latest version and if it sill there, report an issue.

What do you expect by reporting on a resolved issue that you have the same issue without even stating your Airflow version?