authlib already installed, but No module named 'authlib'

[inspirerit]

I running helm use this template
and enabled oauth follwing config:

  enable_oauth: |
     from flask_appbuilder.security.manager import (AUTH_DB, AUTH_OAUTH)
     AUTH_TYPE = AUTH_OAUTH
     OAUTH_PROVIDERS = [
         {
             "name": "google",
             "whitelist": [ os.getenv("OAUTH_WHITELIST_REGEX", "") ],
             "icon": "fa-google",
             "token_key": "access_token",
             "remote_app": {
                 "client_id": os.environ.get("GOOGLE_KEY"),
                 "client_secret": os.environ.get("GOOGLE_SECRET"),
                 "api_base_url": "https://www.googleapis.com/oauth2/v2/",
                 "client_kwargs": {"scope": "email profile"},
                 "request_token_url": None,
                 "access_token_url": "https://accounts.google.com/o/oauth2/token",
                 "authorize_url": "https://accounts.google.com/o/oauth2/auth",
                 "authorize_params": {"hd": os.getenv("OAUTH_HOME_DOMAIN", "")}
             }
         }
     ]

I also preinstall authlib package to superset image
ex. Dockerfile

FROM apache/superset:latest
RUN pip install --no-cache-dir authlib

.But when I run helm to deploy, it still have error:

....
Failed to create app
Traceback (most recent call last):
  File "/app/superset/app.py", line 37, in create_app
    app_initializer.init_app()
  File "/app/superset/initialization/__init__.py", line 473, in init_app
    self.init_app_in_ctx()
  File "/app/superset/initialization/__init__.py", line 418, in init_app_in_ctx
    self.configure_fab()
  File "/app/superset/initialization/__init__.py", line 517, in configure_fab
    appbuilder.init_app(self.superset_app, db.session)
  File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/base.py", line 211, in init_app
    self.sm = self.security_manager_class(self)
  File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/security/sqla/manager.py", line 62, in __init__
    super(SecurityManager, self).__init__(appbuilder)
  File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/security/manager.py", line 263, in __init__
    from authlib.integrations.flask_client import OAuth
ModuleNotFoundError: No module named 'authlib'

So anyone have solution to fix that, I try downgrade authlib to 0.15.x version but still have the same error

[stevehenderson]

I solved the same error by adding authlib==1.2.0 to the boostrapScript section of my-values.yaml (around L43):

Was:

bootstrapScript: |
  #!/bin/bash
  rm -rf /var/lib/apt/lists/* && \
  pip install \
    psycopg2-binary==2.9.1 \
    redis==3.5.3 && \
 .
.
.

After:

bootstrapScript: |
  #!/bin/bash
  rm -rf /var/lib/apt/lists/* && \
  pip install \
    psycopg2-binary==2.9.1 \
    authlib==1.2.0 \
    redis==3.5.3 && \
.
.
.

This is a recent commit to the k8s doc

[LawrenceWarren]

Similar to @stevehenderson, I solved this by adding to supersetWorker.command. From:

command:
  - "/bin/sh"
  - "-c"
  - ". {{ .Values.configMountPath }}/superset_bootstrap.sh; celery --app=superset.tasks.celery_app:app worker"

to:

command:
  - "/bin/sh"
  - "-c"
  - ". {{ .Values.configMountPath }}/superset_bootstrap.sh; pip install authlib==1.2.0; celery --app=superset.tasks.celery_app:app worker"

This is because appending to the bootstrapScript didn't seem to run because this step, somehow. I'm sure with some debugging this could be figured out, but for the time being I'm going with this.

[alephpt]

I fixed this in mine by just adding authlib==1.2.0 to docker/requirements-local.txt

[MartinKChen]

Got the same issue when using 3.0.1 as my base image, with authlib and apache-superset[databricks] installed as extra layer for my deployment.

I tried to docker run the image and execute pip list within the container to ensure required libraries were installed correctly.

But have the issue of "No module named authlib" and Databricks connection did not show up when adding new connections after deployed to my local Kubernetes cluster (i was using kind) with my customized image. And worked as charm after add these libraries under bootstrapScript as @stevehenderson suggested.

Still trying to understand why these libraries does not work, and require to be installed with bootstrap when the image already have them installed.

[MartinKChen]

I refer to the Dockerfile where building dev layer from lean layer. And come out with a new custom Dockerfile, where switch user to root before COPY and RUN pip install, then switch back to the user superset. Now authlib and apache-superset[databricks] are both been installed correctly without additional pip install under bootstrapScript.

Following is my latest Dockerfile:

FROM apache/superset:3.0.1

USER root
COPY ./requirements.txt .
RUN pip install --no-cache-dir -r ./requirements.txt
USER superset

---

Note: I had tried different ways to copy requirements.txt before added switching user, and none of them install extra libraries successfully.

# Rename the file when copy, in case any constraint with the file name "requirements.txt"
COPY ./requirements.txt ./rename.txt
# Change owner when copy the file into image
COPY --chown=superset:superset ./requirements.txt .
# Copy file into requirements directory, instead of under root directory
COPY ./requirements.txt requirements/requirements_custom.txt
# Combine both approaches above
COPY --chown=superset:superset ./requirements.txt requirements/requirements_custom.txt

[belaw]

@alephpt @MartinKChen this issue is about the helm/k8s deployment process, not docker

[rusackas]

Closing this one out, as it seems it's gone quiet/stale, and because there seem to be plenty of folks that reached a solution on the thread. If people are still experiencing this in currently supported versions of Superset (3.1/4.0) just say the word and we can revisit/reopen.

[berlincount]

The workaround is still required with the current version, while it should work out of the box with the stock Docker image.

[1yuv]

Hi @rusackas , this manual step is still needed on 4.0.1.