Skip to content

Problem after closing scheduler before pausing DAG #23381

@Matheus-Homem

Description

@Matheus-Homem

Apache Airflow version

2.2.5 (latest released)

What happened

Coding its the same, I did't change and AirFlow stop working. I guess it is because I closed apache scheduler before pausing the DAG and then deleted the DAG.

`Something bad has happened.

Airflow is used by many users, and it is very likely that others had similar problems and you can easily find
a solution to your problem.

Consider following these steps:

  • gather the relevant information (detailed logs with errors, reproduction steps, details of your deployment)

  • find similar issues using:

  • if you run Airflow on a Managed Service, consider opening an issue using the service support channels

  • if you tried and have difficulty with diagnosing and fixing the problem yourself, consider creating a bug report.
    Make sure however, to include all relevant details and results of your investigation so far.

Python version: 3.8.10
Airflow version: 2.2.5
Node: DESKTOP-BRE4CM7.localdomain

Traceback (most recent call last):
File "/home/matheus/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/home/matheus/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
sqlite3.DatabaseError: database disk image is malformed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/matheus/.local/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/home/matheus/.local/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/matheus/.local/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/matheus/.local/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/home/matheus/.local/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/home/matheus/.local/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functionsrule.endpoint
File "/usr/local/lib/python3.8/dist-packages/airflow/www/auth.py", line 51, in decorated
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/airflow/www/decorators.py", line 108, in view_func
return f(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/airflow/www/decorators.py", line 69, in wrapper
session.add(log)
File "/usr/lib/python3.8/contextlib.py", line 120, in exit
next(self.gen)
File "/usr/local/lib/python3.8/dist-packages/airflow/utils/session.py", line 32, in create_session
session.commit()
File "/home/matheus/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1046, in commit
self.transaction.commit()
File "/home/matheus/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 504, in commit
self._prepare_impl()
File "/home/matheus/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 483, in _prepare_impl
self.session.flush()
File "/home/matheus/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2540, in flush
self._flush(objects)
File "/home/matheus/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2682, in flush
transaction.rollback(capture_exception=True)
File "/home/matheus/.local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in exit
compat.raise
(
File "/home/matheus/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise

raise exception
File "/home/matheus/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush
flush_context.execute()
File "/home/matheus/.local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
rec.execute(self)
File "/home/matheus/.local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 586, in execute
persistence.save_obj(
File "/home/matheus/.local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 239, in save_obj
_emit_insert_statements(
File "/home/matheus/.local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1135, in _emit_insert_statements
result = cached_connections[connection].execute(
File "/home/matheus/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/home/matheus/.local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/home/matheus/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
ret = self._execute_context(
File "/home/matheus/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
self.handle_dbapi_exception(
File "/home/matheus/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in handle_dbapi_exception
util.raise
(
File "/home/matheus/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise

raise exception
File "/home/matheus/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/home/matheus/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.DatabaseError: (sqlite3.DatabaseError) database disk image is malformed
[SQL: INSERT INTO log (dttm, dag_id, task_id, event, execution_date, owner, extra) VALUES (?, ?, ?, ?, ?, ?, ?)]
[parameters: ('2022-04-30 21:47:18.827912', 'treino-04', None, 'tree', None, 'admin', "[('dag_id', 'treino-04')]")]
(Background on this error at: http://sqlalche.me/e/13/4xp6)`

What you think should happen instead

No response

How to reproduce

No response

Operating System

Windows/Ubuntu

Versions of Apache Airflow Providers

No response

Deployment

Virtualenv installation

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions