Skip to content

1.10.12 - > .14 : section/key [google/prompt] not found in config #13154

@gfelot

Description

@gfelot

Apache Airflow version: Going from a working 1.10.12 -> .14

Environment:

What happened:

I have a mix of Terraform/Ansible to handle GCP and the installation on VM. Everything was working good with 1.10.12 and I attemped to update to 1.10.14. The webserver doesn't want to start because of this error :
airflow.exceptions.AirflowConfigException: section/key [google/prompt] not found in config
I can find here in the documentation that part :
https://airflow.apache.org/docs/apache-airflow/1.10.14/security.html#google-authentication
I never had to setup [google/prompt] and I can see it appear in the .13 documentation
It's from this PR : #11890

The documentation is not clear enough for me. So I put what seems to me the less scary : "none".

But now I have an error when I try to init the db :

fatal: [sea-cloud-airflow-applications]: FAILED! => {"msg": "the field 'become' has an invalid value ({{airflow_user}}), and could not be converted to an bool.The error was: The value 'airflow' is not a valid boolean. Valid booleans include: 0, 1, 'no', 'on', 'f', 't', 'off', '0', '1', 'true', 'false', 'n', 'y', 'yes'"}

When I see the journalctl I got this :

   ____________       _____________
  ____    |__( )_________  __/__  /________      __
 ____  /| |_  /__  ___/_  /_ __  /_  __ \_ | /| / /
 ___  ___ |  / _  /   _  __/ _  / / /_/ /_ |/ |/ /
  _/_/  |_/_/  /_/    /_/    /_/  \____/____/|__/
 {__init__.py:50} INFO - Using executor SequentialExecutor
  {dagbag.py:417} INFO - Filling up the DagBag from /home/airflow/airflow/dags
 {dagbag.py:259} ERROR - Failed to import: /usr/local/lib/python3.7/dist-packages/airflow/example_dags/example_subdag_operator.py
 Traceback (most recent call last):
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
     cursor, statement, parameters, context
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/default.py", line 593, in do_execute
     cursor.execute(statement, parameters)
 sqlite3.OperationalError: no such table: slot_pool
 The above exception was the direct cause of the following exception:
 Traceback (most recent call last):
   File "/usr/local/lib/python3.7/dist-packages/airflow/models/dagbag.py", line 256, in process_file
     m = imp.load_source(mod_name, filepath)
   File "/usr/lib/python3.7/imp.py", line 171, in load_source
     module = _load(spec)
   File "<frozen importlib._bootstrap>", line 696, in _load
   File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
   File "<frozen importlib._bootstrap_external>", line 728, in exec_module
   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
   File "/usr/local/lib/python3.7/dist-packages/airflow/example_dags/example_subdag_operator.py", line 51, in <module>
     dag=dag,
   File "/usr/local/lib/python3.7/dist-packages/airflow/utils/db.py", line 74, in wrapper
     return func(*args, **kwargs)
   File "/usr/local/lib/python3.7/dist-packages/airflow/utils/decorators.py", line 98, in wrapper
     result = func(*args, **kwargs)
   File "/usr/local/lib/python3.7/dist-packages/airflow/operators/subdag_operator.py", line 77, in __init__
     .filter(Pool.pool == self.pool)
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/orm/query.py", line 3429, in first
     ret = list(self[0:1])
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/orm/query.py", line 3203, in __getitem__
     return list(res)
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/orm/query.py", line 3535, in __iter__
     return self._execute_and_instances(context)
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/orm/query.py", line 3560, in _execute_and_instances
     result = conn.execute(querycontext.statement, self._params)
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 1011, in execute
     return meth(self, multiparams, params)
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
     return connection._execute_clauseelement(self, multiparams, params)
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 1130, in _execute_clauseelement
     distilled_params,
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 1317, in _execute_context
     e, statement, parameters, cursor, context
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 1511, in _handle_dbapi_exception
     sqlalchemy_exception, with_traceback=exc_info[2], from_=e
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/util/compat.py", line 182, in raise_
     raise exception
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
     cursor, statement, parameters, context
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/default.py", line 593, in do_execute
     cursor.execute(statement, parameters)
 sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: slot_pool
 [SQL: SELECT slot_pool.id AS slot_pool_id, slot_pool.pool AS slot_pool_pool, slot_pool.slots AS slot_pool_slots, slot_pool.description AS slot_pool_description
 FROM slot_pool
 WHERE slot_pool.slots = ? AND slot_pool.pool = ?
  LIMIT ? OFFSET ?]
 [parameters: (1, 'default_pool', 1, 0)]
 (Background on this error at: http://sqlalche.me/e/13/e3q8)
 Traceback (most recent call last):
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
     cursor, statement, parameters, context
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/default.py", line 593, in do_execute
     cursor.execute(statement, parameters)
 sqlite3.OperationalError: no such table: connection
 The above exception was the direct cause of the following exception:
 Traceback (most recent call last):
   File "/usr/local/bin/airflow", line 37, in <module>
     args.func(args)
   File "/usr/local/lib/python3.7/dist-packages/airflow/utils/cli.py", line 81, in wrapper
     return f(*args, **kwargs)
   File "/usr/local/lib/python3.7/dist-packages/airflow/bin/cli.py", line 1179, in webserver
     app = cached_app_rbac(None) if settings.RBAC else cached_app(None)
   File "/usr/local/lib/python3.7/dist-packages/airflow/www/app.py", line 244, in cached_app
     app = create_app(config, testing)
   File "/usr/local/lib/python3.7/dist-packages/airflow/www/app.py", line 99, in create_app
     from airflow.www import views
   File "/usr/local/lib/python3.7/dist-packages/airflow/www/views.py", line 2504, in <module>
     class ChartModelView(wwwutils.DataProfilingMixin, AirflowModelView):
   File "/usr/local/lib/python3.7/dist-packages/airflow/www/views.py", line 2595, in ChartModelView
     'conn_id': _connection_ids()
   File "/usr/local/lib/python3.7/dist-packages/airflow/utils/db.py", line 74, in wrapper
     return func(*args, **kwargs)
   File "/usr/local/lib/python3.7/dist-packages/airflow/www/views.py", line 2501, in _connection_ids
     .group_by(Connection.conn_id))]
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/orm/query.py", line 3535, in __iter__
     return self._execute_and_instances(context)
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/orm/query.py", line 3560, in _execute_and_instances
     result = conn.execute(querycontext.statement, self._params)
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 1011, in execute
     return meth(self, multiparams, params)
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
     return connection._execute_clauseelement(self, multiparams, params)
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 1130, in _execute_clauseelement
     distilled_params,
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 1317, in _execute_context
     e, statement, parameters, cursor, context
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 1511, in _handle_dbapi_exception
     sqlalchemy_exception, with_traceback=exc_info[2], from_=e
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/util/compat.py", line 182, in raise_
     raise exception
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
     cursor, statement, parameters, context
   File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/engine/default.py", line 593, in do_execute
     cursor.execute(statement, parameters)
 sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: connection
 [SQL: SELECT connection.conn_id AS connection_conn_id
 FROM connection GROUP BY connection.conn_id]
 (Background on this error at: http://sqlalche.me/e/13/e3q8)
 airflow-webserver.service: Main process exited, code=exited, status=1/FAILURE
 airflow-webserver.service: Failed with result 'exit-code'.
 airflow-webserver.service: Service RestartSec=5s expired, scheduling restart.
 airflow-webserver.service: Scheduled restart job, restart counter is at 358.
 Stopped Airflow webserver daemon.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions