Webserver IntegrityError When Loaded

[ahipp13]

Apache Airflow version

2.4.3

What happened

We just upgraded from Airflow 2.2.5 to 2.4.3. When using Azure OAuth, whenever you load the webserver, it loads it with an error and in a weird looking format:

It loads it in a weird different format every time you hit the refresh button, and has the same error every time (full logs down below):

airflow-web sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "session_session_id_key"
airflow-web DETAIL: Key (session_id)=(7582aee0-4289-4152-99d2-918c98ff64d6) already exists.
airflow-web
airflow-web [SQL: INSERT INTO session (session_id, data, expiry) VALUES (%(session_id)s, %(data)s, %(expiry)s) RETURNING session.id]
airflow-web [parameters: {'session_id': '7582aee0-4289-4152-99d2-918c98ff64d6', 'data': <psycopg2.extensions.Binary object at 0x7fbd53ad71b0>, 'expiry': datetime.datetime(2022, 11, 23, 16, 5, 59, 467562, tzinfo=datetime.timezone.utc)}]
airflow-web (Background on this error at: https://sqlalche.me/e/14/gkpj)
airflow-web

What you think should happen instead

The webserver should display the normal login page without any errors. Here are the full logs from the kubernetes pod:

webserver.logs

airflow-web 127.0.0.1 - - [23/Nov/2022:15:35:59 +0000] "GET /login/ HTTP/1.1" 200 16330 "https://login.microsoftonline.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 Edg/106.0.1370.52" airflow-web 127.0.0.1 - - [23/Nov/2022:15:35:59 +0000] "GET /static/appbuilder/css/font-awesome.min.css HTTP/1.1" 200 0 "https://url/login/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 Edg/106.0.1370.52" airflow-web [2022-11-23 15:35:59,423] {app.py:1741} ERROR - Exception on /static/appbuilder/css/bootstrap.min.css [GET] airflow-web Traceback (most recent call last): airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context airflow-web self.dialect.do_execute( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute airflow-web cursor.execute(statement, parameters) airflow-web psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "session_session_id_key" airflow-web DETAIL: Key (session_id)=(7582aee0-4289-4152-99d2-918c98ff64d6) already exists. airflow-web airflow-web return super().save_session(*args, **kwargs) airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask_session/sessions.py", line 578, in save_session airflow-web self.db.session.commit() airflow-web File "", line 2, in commit airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1428, in commit airflow-web self._transaction.commit(_to_root=self.future) airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 829, in commit airflow-web self._prepare_impl() airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl airflow-web self.session.flush() airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3345, in flush airflow-web self._flush(objects) airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3485, in flush airflow-web transaction.rollback(capture_exception=True) airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit airflow-web compat.raise( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise airflow-web raise exception airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3445, in _flush airflow-web flush_context.execute() airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute airflow-web rec.execute(self) airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute airflow-web util.preloaded.orm_persistence.save_obj( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 244, in save_obj airflow-web _emit_insert_statements( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1221, in _emit_insert_statements airflow-web result = connection._execute_20( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1614, in _execute_20 airflow-web return meth(self, args_10style, kwargs_10style, execution_options) airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection airflow-web return connection._execute_clauseelement( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1481, in _execute_clauseelement airflow-web ret = self._execute_context( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1845, in _execute_context airflow-web self.handle_dbapi_exception( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2026, in handle_dbapi_exception airflow-web util.raise( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise airflow-web raise exception airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context airflow-web self.dialect.do_execute( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute airflow-web cursor.execute(statement, parameters) airflow-web sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "session_session_id_key" airflow-web DETAIL: Key (session_id)=(7582aee0-4289-4152-99d2-918c98ff64d6) already exists. airflow-web airflow-web [SQL: INSERT INTO session (session_id, data, expiry) VALUES (%(session_id)s, %(data)s, %(expiry)s) RETURNING session.id] airflow-web [parameters: {'session_id': '7582aee0-4289-4152-99d2-918c98ff64d6', 'data': , 'expiry': datetime.datetime(2022, 11, 23, 16, 5, 59, 467562, tzinfo=datetime.timezone.utc)}] airflow-web (Background on this error at: https://sqlalche.me/e/14/gkpj) airflow-web airflow-web During handling of the above exception, another exception occurred: airflow-web airflow-web Traceback (most recent call last): airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 1844, in finalize_request airflow-web response = self.process_response(response) airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 2340, in process_response airflow-web self.session_interface.save_session(self, ctx.session, response) airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/airflow/www/session.py", line 33, in save_session airflow-web return super().save_session(*args, **kwargs) airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask_session/sessions.py", line 553, in save_session airflow-web saved_session = self.sql_session_model.query.filter_by( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2810, in first airflow-web return self.limit(1)._iter().first() airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2894, in _iter airflow-web result = self.session.execute( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1688, in execute airflow-web conn = self._connection_for_bind(bind) airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1529, in _connection_for_bind airflow-web return self._transaction._connection_for_bind( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 721, in _connection_for_bind airflow-web self._assert_active() airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 601, in _assert_active airflow-web raise sa_exc.PendingRollbackError( airflow-web sqlalchemy.exc.PendingRollbackError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "session_session_id_key" airflow-web DETAIL: Key (session_id)=(7582aee0-4289-4152-99d2-918c98ff64d6) already exists. airflow-web airflow-web [SQL: INSERT INTO session (session_id, data, expiry) VALUES (%(session_id)s, %(data)s, %(expiry)s) RETURNING session.id] airflow-web [parameters: {'session_id': '7582aee0-4289-4152-99d2-918c98ff64d6', 'data': , 'expiry': datetime.datetime(2022, 11, 23, 16, 5, 59, 467562, tzinfo=datetime.timezone.utc)}] airflow-web (Background on this error at: https://sqlalche.me/e/14/gkpj) (Background on this error at: https://sqlalche.me/e/14/7s2a) airflow-web 127.0.0.1 - - [23/Nov/2022:15:35:59 +0000] "GET /static/appbuilder/css/flags/flags16.css HTTP/1.1" 500 1575 "https://url/login/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 Edg/106.0.1370.52" airflow-web 127.0.0.1 - - [23/Nov/2022:15:35:59 +0000] "GET /static/appbuilder/css/ab.css HTTP/1.1" 500 1575 "https://url/login/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 Edg/106.0.1370.52" airflow-web 127.0.0.1 - - [23/Nov/2022:15:35:59 +0000] "GET /static/dist/airflowDefaultTheme.731e57571b52cca4350d.css HTTP/1.1" 200 0 "https://url/login/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 Edg/106.0.1370.52" airflow-web [2022-11-23 15:35:59,534] {app.py:1741} ERROR - Exception on /static/dist/materialIcons.f9559e4953177b8b9a4a.css [GET] airflow-web Traceback (most recent call last): airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context airflow-web self.dialect.do_execute( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute airflow-web cursor.execute(statement, parameters) airflow-web psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "session_session_id_key" airflow-web DETAIL: Key (session_id)=(7582aee0-4289-4152-99d2-918c98ff64d6) already exists. airflow-web airflow-web airflow-web The above exception was the direct cause of the following exception: airflow-web airflow-web Traceback (most recent call last): airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 2525, in wsgi_app airflow-web response = self.full_dispatch_request() airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 1823, in full_dispatch_request airflow-web return self.finalize_request(rv) airflow-web return meth(self, args_10style, kwargs_10style, execution_options) airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection airflow-web return connection._execute_clauseelement( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1481, in _execute_clauseelement airflow-web ret = self._execute_context( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1845, in _execute_context airflow-web self.handle_dbapi_exception( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2026, in handle_dbapi_exception airflow-web util.raise( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise airflow-web raise exception airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context airflow-web self.dialect.do_execute( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute airflow-web cursor.execute(statement, parameters) airflow-web sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "session_session_id_key" airflow-web DETAIL: Key (session_id)=(7582aee0-4289-4152-99d2-918c98ff64d6) already exists. airflow-web airflow-web [SQL: INSERT INTO session (session_id, data, expiry) VALUES (%(session_id)s, %(data)s, %(expiry)s) RETURNING session.id] airflow-web [parameters: {'session_id': '7582aee0-4289-4152-99d2-918c98ff64d6', 'data': , 'expiry': datetime.datetime(2022, 11, 23, 16, 5, 59, 574672, tzinfo=datetime.timezone.utc)}] airflow-web (Background on this error at: https://sqlalche.me/e/14/gkpj) airflow-web [2022-11-23 15:35:59,591] {app.py:1741} ERROR - Exception on /static/dist/loadingDots.5da42d00b5455806e709.css [GET] airflow-web Traceback (most recent call last): airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context airflow-web self.dialect.do_execute( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute airflow-web cursor.execute(statement, parameters) airflow-web psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "session_session_id_key" airflow-web DETAIL: Key (session_id)=(7582aee0-4289-4152-99d2-918c98ff64d6) already exists. airflow-web airflow-web airflow-web The above exception was the direct cause of the following exception: airflow-web airflow-web Traceback (most recent call last): airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 2525, in wsgi_app airflow-web response = self.full_dispatch_request() airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 1823, in full_dispatch_request airflow-web return self.finalize_request(rv) airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 1844, in finalize_request airflow-web response = self.process_response(response) airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 2340, in process_response airflow-web self.session_interface.save_session(self, ctx.session, response) airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/airflow/www/session.py", line 33, in save_session airflow-web return super().save_session(*args, **kwargs) airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask_session/sessions.py", line 578, in save_session airflow-web self.db.session.commit() airflow-web File "", line 2, in commit airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1428, in commit airflow-web self._transaction.commit(_to_root=self.future) airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 829, in commit airflow-web self._prepare_impl() airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl airflow-web self.session.flush() airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3345, in flush airflow-web self._flush(objects) airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3485, in flush airflow-web transaction.rollback(capture_exception=True) airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit airflow-web compat.raise( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise airflow-web raise exception airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3445, in _flush airflow-web flush_context.execute() airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute airflow-web rec.execute(self) airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute airflow-web util.preloaded.orm_persistence.save_obj( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 244, in save_obj airflow-web _emit_insert_statements( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1221, in _emit_insert_statements airflow-web result = connection._execute_20( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1614, in _execute_20 airflow-web return meth(self, args_10style, kwargs_10style, execution_options) airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection airflow-web return connection._execute_clauseelement( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1481, in _execute_clauseelement airflow-web ret = self._execute_context( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1845, in _execute_context airflow-web self.handle_dbapi_exception( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2026, in handle_dbapi_exception airflow-web util.raise( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise airflow-web raise exception airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context airflow-web self.dialect.do_execute( airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute airflow-web cursor.execute(statement, parameters) airflow-web sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "session_session_id_key" airflow-web DETAIL: Key (session_id)=(7582aee0-4289-4152-99d2-918c98ff64d6) already exists.

How to reproduce

Down below is a copy of our webserver_config.py. We are currently running Airflow 2.4.3 on Kubernetes with the Airflow Community helm chart version 8.6.1 (located here: https://github.com/airflow-helm/charts). We are also using a postgres external database as our metadata db.

from flask_appbuilder.security.manager import AUTH_OAUTH
from airflow.www.security import AirflowSecurityManager
import logging
from typing import Dict, Any, List, Union
import os
import sys

#Add this as a module to pythons path
sys.path.append('/opt/airflow')

log = logging.getLogger(__name__)
log.setLevel(os.getenv("AIRFLOW__LOGGING__FAB_LOGGING_LEVEL", "DEBUG"))

class AzureCustomSecurity(AirflowSecurityManager):
    # In this example, the oauth provider == 'azure'.
    # If you ever want to support other providers, see how it is done here:
    # https://github.com/dpgaspar/Flask-AppBuilder/blob/master/flask_appbuilder/security/manager.py#L550
    def get_oauth_user_info(self, provider, resp):
        # Creates the user info payload from Azure.
        # The user previously allowed your app to act on their behalf,
        #   so now we can query the user and teams endpoints for their data.
        # Username and team membership are added to the payload and returned to FAB.
        if provider == "azure":
            log.debug("Azure response received : {0}".format(resp))
            id_token = resp["id_token"]
            log.debug(str(id_token))
            me = self._azure_jwt_token_parse(id_token)
            log.debug("Parse JWT token : {0}".format(me))
            return {
                "name": me.get("name", ""),
                "email": me["upn"],
                "first_name": me.get("given_name", ""),
                "last_name": me.get("family_name", ""),
                "id": me["oid"],
                "username": me["oid"],
                "role_keys": me.get("roles", []),
            }

# Adding this in because if not the redirect url will start with http and we want https
os.environ["AIRFLOW__WEBSERVER__ENABLE_PROXY_FIX"] = "True"
WTF_CSRF_ENABLED = False
CSRF_ENABLED = False
AUTH_TYPE = AUTH_OAUTH
AUTH_ROLES_SYNC_AT_LOGIN = True  # Checks roles on every login
# Make sure to replace this with the path to your security manager class
FAB_SECURITY_MANAGER_CLASS = "webserver_config.AzureCustomSecurity"
# a mapping from the values of `userinfo["role_keys"]` to a list of FAB roles
AUTH_ROLES_MAPPING = {
    "airflow_dev_admin": ["Admin"],
    "airflow_dev_op": ["Op"],
    "airflow_dev_user": ["User"],
    "airflow_dev_viewer": ["Viewer"]
    }
# force users to re-auth after 30min of inactivity (to keep roles in sync)
PERMANENT_SESSION_LIFETIME = 1800
# If you wish, you can add multiple OAuth providers.
OAUTH_PROVIDERS = [
    {
        "name": "azure",
        "icon": "fa-windows",
        "token_key": "access_token",
        "remote_app": {
            "client_id": "CLIENT_ID",
            "client_secret": 'AZURE_DEV_CLIENT_SECRET',
            "api_base_url": "https://login.microsoftonline.com/TENANT_ID",
            "request_token_url": None,
            'request_token_params': {
                'scope': 'openid email profile'
            },
            "access_token_url": "https://login.microsoftonline.com/TENANT_ID/oauth2/v2.0/token",
            "access_token_params": {
                'scope': 'openid email profile'
            },
            "authorize_url": "https://login.microsoftonline.com/TENANT_ID/oauth2/v2.0/authorize",
            "authorize_params": {
                'scope': 'openid email profile',
            },
            'jwks_uri':'https://login.microsoftonline.com/common/discovery/v2.0/keys',
        },
    },
]

Operating System

Debian GNU/Linux 11 (bullseye)

Versions of Apache Airflow Providers

apache-airflow-providers-amazon==6.0.0
apache-airflow-providers-celery==3.0.0
apache-airflow-providers-cncf-kubernetes==4.4.0
apache-airflow-providers-common-sql==1.2.0
apache-airflow-providers-docker==3.2.0
apache-airflow-providers-elasticsearch==4.2.1
apache-airflow-providers-ftp==3.1.0
apache-airflow-providers-google==8.4.0
apache-airflow-providers-grpc==3.0.0
apache-airflow-providers-hashicorp==3.1.0
apache-airflow-providers-http==4.0.0
apache-airflow-providers-imap==3.0.0
apache-airflow-providers-microsoft-azure==4.3.0
apache-airflow-providers-mysql==3.2.1
apache-airflow-providers-odbc==3.1.2
apache-airflow-providers-postgres==5.2.2
apache-airflow-providers-redis==3.0.0
apache-airflow-providers-sendgrid==3.0.0
apache-airflow-providers-sftp==4.1.0
apache-airflow-providers-slack==6.0.0
apache-airflow-providers-sqlite==3.2.1
apache-airflow-providers-ssh==3.2.0

Deployment

Other 3rd-party Helm chart

Deployment details

We are currently running Airflow 2.4.3 on Kubernetes with the Airflow Community helm chart version 8.6.1 (located here: https://github.com/airflow-helm/charts). We are also using a postgres external database as our metadata db.

Anything else

There is a workaround for this issue. If you set AIRFLOW__WEBSERVER__WORKERS: "1", then the errors go away. This problem happens anytime you load the webserver. New to posting issues so any other information needed please let me know.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[potiuk]

I do not thin the issue is with with the integrity key - this is a result of a problematic integration with Azure. The "azure" button is expected - this comes from Flask Application Builder and it has been added in one of the recent FAB versions.

I guess the duplicate session is harmless and comes from a redirection loop that results from an invalid configuration of Azure. Some of the configuration has changed in the new FAB - it had breaking changes when it comes to configuration and when you run from 2.2.5 to 2.3.4 you also change FAB:

• In 2.4.3 it is FAB 4.1.4: https://github.com/apache/airflow/blob/constraints-2.4.3/constraints-3.9.txt#L16
• In 2.2.5 it was FAB 3.4.5 https://github.com/apache/airflow/blob/constraints-2.2.5/constraints-3.9.txt#L16

This was a major upgrade in FAB and it had some breaking changes or fixing some of the invalid configuration. I suggest to look in the FAB documentation and see if you can make your configuration for that from scratch https://flask-appbuilder.readthedocs.io/en/latest/security.html

Also you can read FAB's release notes - they might help you to migrate.

There is also a possibility that you are hitting some other authlib and other dependencies problems. see #27973 which you also opened.

Unfortunately, we are not specialists in the OAuth - this comes from FAB and they will likely be able to help better with your issue. Converting into a discussion.