Skip to content

Custom webserver_config.py and getting "ModuleNotFoundError: No module named 'authlib'" #17542

@kn6405

Description

@kn6405

I pushed out a custom webserver_config.py to enable oauth authentication for the webserver pod:

Apache Airflow version: 2.1.2
Kubernetes version: v1.18.10
Environment: Azure AKS using the official apache/airflow helm chart (1.1.0)

Running celery executor. With default webserver_config.py everything works fine. Not really sure what this error means and how to resolve. Or if I'm setting things up incorrectly causing this. I'm hoping to get Azure AZ App working with oauth2 then doing some custom role mapping (AUTH_ROLES_MAPPING).

When the pod starts I now get the error:

____________ _____________
____ |__( )_________ __/__ /________ __
____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
_/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
[2021-08-10 21:52:00,980] {dagbag.py:496} INFO - Filling up the DagBag from /dev/null
Traceback (most recent call last):
File "/home/airflow/.local/bin/airflow", line 8, in <module>
sys.exit(main())
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/__main__.py", line 40, in main
args.func(args)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/cli.py", line 91, in wrapper
return f(*args, **kwargs)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/cli/commands/webserver_command.py", line 360, in webserver
app = cached_app(None)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/www/app.py", line 146, in cached_app
app = create_app(config=config, testing=testing)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/www/app.py", line 123, in create_app
init_appbuilder(flask_app)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/www/extensions/init_appbuilder.py", line 51, in init_appbuilder
update_perms=conf.getboolean('webserver', 'UPDATE_FAB_PERMS'),
File "/home/airflow/.local/lib/python3.6/site-packages/flask_appbuilder/base.py", line 148, in __init__
self.init_app(app, session)
File "/home/airflow/.local/lib/python3.6/site-packages/flask_appbuilder/base.py", line 202, in init_app
self.sm = self.security_manager_class(self)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/www/security.py", line 186, in __init__
super().__init__(appbuilder)
File "/home/airflow/.local/lib/python3.6/site-packages/flask_appbuilder/security/sqla/manager.py", line 52, in __init__
super(SecurityManager, self).__init__(appbuilder)
File "/home/airflow/.local/lib/python3.6/site-packages/flask_appbuilder/security/manager.py", line 256, in __init__
from authlib.integrations.flask_client import OAuth
ModuleNotFoundError: No module named 'authlib'

My webserver_config.py:

import os
from logging import getLogger
from flask_appbuilder.security.manager import AUTH_OAUTH
from airflow.configuration import conf

basedir = os.path.abspath(os.path.dirname(__file__))

AZURE_APPLICATION_ID = os.environ.get("AZURE_APPLICATION_ID")
AZURE_SECRET = os.environ.get("AZURE_SECRET")
AZURE_TENANT_ID = os.environ.get("AZURE_TENANT_ID")

WTF_CSRF_ENABLED = True
AUTH_TYPE = AUTH_OAUTH
SQLALCHEMY_DATABASE_URI = conf.get('core', 'SQL_ALCHEMY_CONN')

log = getLogger(__name__)

# Azure OAuth
OAUTH_PROVIDERS = [{
  "name":"azure",
    "token_key":"access_token",
    "icon":"fa-windows",
        "remote_app": {
           "client_id": {AZURE_APPLICATION_ID},
           "client_secret": {AZURE_SECRET},
           "api_base_url": "https://login.microsoftonline.com/{AZURE_TENANT_ID}/oauth2",
           "client_kwargs": {
               "scope": "User.read name preferred_username email profile upn",
               "resource": {AZURE_APPLICATION_ID},
           },
           "request_token_url": None,
           "access_token_url": "https://login.microsoftonline.com/{AZURE_TENANT_ID}/oauth2/token",
           "authorize_url": "https://login.microsoftonline.com/{AZURE_TENANT_ID}/oauth2/authorize",
        }
}]

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind:bugThis is a clearly a bug

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions