Strange SQL related issue

[chriske]

Hi All!

I'm designing an Airflow platform in an enterprise environment. It runs on OpenShift, within containers. Everything was worked fine, but we had to start to migrate to an EDB database, outside Kubernetes. This is OK, EDB is just a "fancy" Postgres. We made our compatibility test, and everything went well. I already integrated EDB in our Sandbox environment. It works just fine since 2 weeks. Without any issue. Airflow is in the latest release btw.

Yesterday I started the EDB integaration of our next environment. This EDB is a completely different instance (different DB, VM, everything). And I cannot tell why, but all of the Airflow components (scheduler, triggerer, webserver) are constantly crashing with similar stacktrace like this:

File "/home/airflow/.local/lib/python3.8/site-packages/airflow/jobs/triggerer_job.py", line 106, in _execute
self._run_trigger_loop()
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/jobs/triggerer_job.py", line 135, in _run_trigger_loop
self.heartbeat(only_if_necessary=True)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/jobs/base_job.py", line 204, in heartbeat
session.merge(self)
File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2955, in merge
return self._merge(
File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3035, in _merge
merged = self.get(
File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2768, in get
return self._get_impl(
File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2874, in _get_impl
return db_load_fn(
File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 541, in load_on_pk_identity
return result.one()
File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/result.py", line 1407, in one
return self._only_one_row(
File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/result.py", line 558, in _only_one_row
row = onerow(hard_close=True)
File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/result.py", line 1271, in _fetchone_impl
return self._real_result._fetchone_impl(hard_close=hard_close)
File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/result.py", line 1674, in _fetchone_impl
row = next(self.iterator, _NO_ROW)
File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 147, in chunks
fetch = cursor._raw_all_rows()
File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/result.py", line 391, in _raw_all_rows
rows = self._fetchall_impl()
File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/cursor.py", line 1803, in _fetchall_impl
return self.cursor_strategy.fetchall(self, self.cursor)
File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/cursor.py", line 981, in fetchall
self.handle_exception(result, dbapi_cursor, e)
File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/cursor.py", line 941, in handle_exception
result.connection.handle_dbapi_exception(
File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2030, in handle_dbapi_exception
util.raise(exc_info[1], with_traceback=exc_info[2])
File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise
raise exception
File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/cursor.py", line 977, in fetchall
rows = dbapi_cursor.fetchall()
ValueError: month must be in 1..12

It is clear, that this issue has to do something with the Database and the heartbeat, but I can't figure it out exactly what. I asked the DBAs to differentiate the settings of the two databases, but they found nothing. Is there anyone who maybe familiar with this issue and can help me?

Thanks in advance!

[chriske]

Okay, now we figured out, The datestyle of the database was the issue. It was setted something that represented a date like this: 21-OCT-22. This was the issue with the month. The solution was to adjust the database's datesytle to ISO.