Skip to content

FUNCTION airflow.JSON_VALID with airflow version 2.0.0  #13809

@SaithZhang

Description

@SaithZhang

Apache Airflow version:
2.0.0

Kubernetes version (if you are using kubernetes) (use kubectl version):

Environment:
Python 3.7.3
Linux Version

uname -r
3.10.0-693.21.1.el7.x86_64

cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"

mysql instance version
5.6.26-log

  • Cloud provider or hardware configuration:
  • OS (e.g. from /etc/os-release):
  • Kernel (e.g. uname -a):
  • Install tools:
  • Others:

What happened:

i use command to install airflow
pip install apache-airflow pip install apache-airflow[mysql]
then change the airflow.cfg
sql_alchemy_conn = mysql+pymysql://airflow:xxx@mydomain:3653/airflow
then execute the command
airflow db init
the error occured

INFO [alembic.runtime.migration] Running upgrade 6e96a59344a4 -> d38e04c12aa2, add serialized_dag table
Traceback (most recent call last):
File "/data/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1278, in _execute_context
cursor, statement, parameters, context
File "/data/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
cursor.execute(statement, parameters)
File "/data/anaconda3/lib/python3.7/site-packages/pymysql/cursors.py", line 170, in execute
result = self._query(query)
File "/data/anaconda3/lib/python3.7/site-packages/pymysql/cursors.py", line 328, in _query
conn.query(q)
File "/data/anaconda3/lib/python3.7/site-packages/pymysql/connections.py", line 517, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/data/anaconda3/lib/python3.7/site-packages/pymysql/connections.py", line 732, in _read_query_result
result.read()
File "/data/anaconda3/lib/python3.7/site-packages/pymysql/connections.py", line 1075, in read
first_packet = self.connection._read_packet()
File "/data/anaconda3/lib/python3.7/site-packages/pymysql/connections.py", line 684, in _read_packet
packet.check_error()
File "/data/anaconda3/lib/python3.7/site-packages/pymysql/protocol.py", line 220, in check_error
err.raise_mysql_exception(self._data)
File "/data/anaconda3/lib/python3.7/site-packages/pymysql/err.py", line 109, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.InternalError: (1305, 'FUNCTION airflow.JSON_VALID does not exist')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/data/anaconda3/bin/airflow", line 10, in
sys.exit(main())
File "/data/anaconda3/lib/python3.7/site-packages/airflow/main.py", line 40, in main
args.func(args)
File "/data/anaconda3/lib/python3.7/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
File "/data/anaconda3/lib/python3.7/site-packages/airflow/cli/commands/db_command.py", line 31, in initdb
db.initdb()
File "/data/anaconda3/lib/python3.7/site-packages/airflow/utils/db.py", line 549, in initdb
upgradedb()
File "/data/anaconda3/lib/python3.7/site-packages/airflow/utils/db.py", line 688, in upgradedb
command.upgrade(config, 'heads')
File "/data/anaconda3/lib/python3.7/site-packages/alembic/command.py", line 298, in upgrade
script.run_env()
File "/data/anaconda3/lib/python3.7/site-packages/alembic/script/base.py", line 489, in run_env
util.load_python_file(self.dir, "env.py")
File "/data/anaconda3/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 98, in load_python_file
module = load_module_py(module_id, path)
File "/data/anaconda3/lib/python3.7/site-packages/alembic/util/compat.py", line 184, in load_module_py
spec.loader.exec_module(module)
File "", line 728, in exec_module
File "", line 219, in _call_with_frames_removed
File "/data/anaconda3/lib/python3.7/site-packages/airflow/migrations/env.py", line 108, in
run_migrations_online()
File "/data/anaconda3/lib/python3.7/site-packages/airflow/migrations/env.py", line 102, in run_migrations_online
context.run_migrations()
File "", line 8, in run_migrations
File "/data/anaconda3/lib/python3.7/site-packages/alembic/runtime/environment.py", line 846, in run_migrations
self.get_context().run_migrations(**kw)
File "/data/anaconda3/lib/python3.7/site-packages/alembic/runtime/migration.py", line 522, in run_migrations
step.migration_fn(**kw)
File "/data/anaconda3/lib/python3.7/site-packages/airflow/migrations/versions/d38e04c12aa2_add_serialized_dag_table.py", line 46, in upgrade
conn.execute("SELECT JSON_VALID(1)").fetchone()
File "/data/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1006, in execute
return self.execute_text(object, multiparams, params)
File "/data/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1181, in _execute_text
parameters,
File "/data/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1318, in execute_context
e, statement, parameters, cursor, context
File "/data/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1512, in handle_dbapi_exception
sqlalchemy_exception, with_traceback=exc_info[2], from
=e
File "/data/anaconda3/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 178, in raise

raise exception
File "/data/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1278, in _execute_context
cursor, statement, parameters, context
File "/data/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
cursor.execute(statement, parameters)
File "/data/anaconda3/lib/python3.7/site-packages/pymysql/cursors.py", line 170, in execute
result = self._query(query)
File "/data/anaconda3/lib/python3.7/site-packages/pymysql/cursors.py", line 328, in _query
conn.query(q)
File "/data/anaconda3/lib/python3.7/site-packages/pymysql/connections.py", line 517, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/data/anaconda3/lib/python3.7/site-packages/pymysql/connections.py", line 732, in _read_query_result
result.read()
File "/data/anaconda3/lib/python3.7/site-packages/pymysql/connections.py", line 1075, in read
first_packet = self.connection._read_packet()
File "/data/anaconda3/lib/python3.7/site-packages/pymysql/connections.py", line 684, in _read_packet
packet.check_error()
File "/data/anaconda3/lib/python3.7/site-packages/pymysql/protocol.py", line 220, in check_error
err.raise_mysql_exception(self._data)
File "/data/anaconda3/lib/python3.7/site-packages/pymysql/err.py", line 109, in raise_mysql_exception
raise errorclass(errno, errval)
sqlalchemy.exc.InternalError: (pymysql.err.InternalError) (1305, 'FUNCTION airflow.JSON_VALID does not exist')
[SQL: SELECT JSON_VALID(1)]
(Background on this error at: http://sqlalche.me/e/13/2j85)

What you expected to happen:
the db should init success

i use mysql version 5.6.26, which is deployed in company env, it's hard to upgrade to 5.7+.
i wonder is there a workarond to use airflow 2.0.0 with mysql 5.6.26, can i bypass the JSON_VALID function?

How to reproduce it:

install airflow version 2.0.0 with mysql instance 5.6.26 then init db should reproduce it.

Anything else we need to know:

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions