OIDC settings in docker-compose file throw error in datahub frontend #7733

jakobhanna · 2023-03-31T13:47:31Z

Describe the bug
Using the compose quickstart file with OIDC customization the datahub frontend container throws an error.

To Reproduce
Steps to reproduce the behavior:

Go to docker/quickstart/ and configure docker-compose-without-neo4j.quickstart
Change the OIDC settings
Start datahub with python3 -m datahub docker quickstart --quickstart-compose-file docker/quickstart/docker-compose-without-neo4j.quickstart.yml
Show the logs

Expected behavior
Datahub frontend should work as expected. Without OIDC settings the service starts normal.

Desktop (please complete the following information):

OS: [Ubuntu 22.04]

Additional context
Here the error log:

datahub-frontend-react        | WARNING: An illegal reflective access operation has occurred
datahub-frontend-react        | WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$1 (file:/datahub-frontend/lib/com.google.inject-guice-4.2.3.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
datahub-frontend-react        | WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$1
datahub-frontend-react        | WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
datahub-frontend-react        | WARNING: All illegal access operations will be denied in a future release
datahub-frontend-react        | Oops, cannot start the server.
datahub-frontend-react        | com.google.inject.CreationException: Unable to create injector, see the following errors:
datahub-frontend-react        | 
datahub-frontend-react        | 1) No implementation for play.cache.SyncCacheApi was bound.
datahub-frontend-react        |   at auth.AuthModule.configure(AuthModule.java:81) (via modules: com.google.inject.util.Modules$OverrideModule -> auth.AuthModule)
datahub-frontend-react        | 
datahub-frontend-react        | 1 error
datahub-frontend-react        |         at com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:554)
datahub-frontend-react        |         at com.google.inject.internal.InternalInjectorCreator.initializeStatically(InternalInjectorCreator.java:161)
datahub-frontend-react        |         at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:108)
datahub-frontend-react        |         at com.google.inject.Guice.createInjector(Guice.java:87)
datahub-frontend-react        |         at com.google.inject.Guice.createInjector(Guice.java:78)
datahub-frontend-react        |         at play.api.inject.guice.GuiceBuilder.injector(GuiceInjectorBuilder.scala:200)
datahub-frontend-react        |         at play.inject.guice.GuiceBuilder.injector(GuiceBuilder.java:211)
datahub-frontend-react        |         at play.inject.guice.GuiceApplicationBuilder.build(GuiceApplicationBuilder.java:121)
datahub-frontend-react        |         at play.inject.guice.GuiceApplicationLoader.load(GuiceApplicationLoader.java:32)
datahub-frontend-react        |         at play.api.ApplicationLoader$JavaApplicationLoaderAdapter$1.load(ApplicationLoader.scala:181)
datahub-frontend-react        |         at play.core.server.ProdServerStart$.start(ProdServerStart.scala:53)
datahub-frontend-react        |         at play.core.server.ProdServerStart$.main(ProdServerStart.scala:29)
datahub-frontend-react        |         at play.core.server.ProdServerStart.main(ProdServerStart.scala)
datahub-frontend-react exited with code 255

The text was updated successfully, but these errors were encountered:

jakobhanna · 2023-03-31T13:56:22Z

with those OIDC settings

networks:
  default:
    name: datahub_network
services:
  broker:
    container_name: broker
    depends_on:
      - zookeeper
    environment:
      - KAFKA_BROKER_ID=1
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092
      - KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1
      - KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS=0
      - KAFKA_HEAP_OPTS=-Xms256m -Xmx256m
      - KAFKA_CONFLUENT_SUPPORT_METRICS_ENABLE=false
    hostname: broker
    image: confluentinc/cp-kafka:7.2.2
    ports:
      - ${DATAHUB_MAPPED_KAFKA_BROKER_PORT:-9092}:9092
  datahub-actions:
    depends_on:
      - datahub-gms
    environment:
      - DATAHUB_GMS_HOST=datahub-gms
      - DATAHUB_GMS_PORT=8080
      - DATAHUB_GMS_PROTOCOL=http
      - DATAHUB_SYSTEM_CLIENT_ID=__datahub_system
      - DATAHUB_SYSTEM_CLIENT_SECRET=JohnSnowKnowsNothing
      - KAFKA_BOOTSTRAP_SERVER=broker:29092
      - KAFKA_PROPERTIES_SECURITY_PROTOCOL=PLAINTEXT
      - METADATA_AUDIT_EVENT_NAME=MetadataAuditEvent_v4
      - METADATA_CHANGE_LOG_VERSIONED_TOPIC_NAME=MetadataChangeLog_Versioned_v1
      - SCHEMA_REGISTRY_URL=http://schema-registry:8081
    hostname: actions
    image: acryldata/datahub-actions:${ACTIONS_VERSION:-head}
    restart: on-failure:5
  datahub-frontend-react:
    container_name: datahub-frontend-react
    depends_on:
      - datahub-gms
    environment:
      - DATAHUB_GMS_HOST=datahub-gms
      - DATAHUB_GMS_PORT=8080
      - DATAHUB_SECRET=YouKnowNothing
      - DATAHUB_APP_VERSION=1.0
      - DATAHUB_PLAY_MEM_BUFFER_SIZE=10MB
      - JAVA_OPTS=-Xms512m -Xmx512m -Dhttp.port=9002 -Dconfig.file=datahub-frontend/conf/application.conf -Djava.security.auth.login.config=datahub-frontend/conf/jaas.conf -Dlogback.configurationFile=datahub-frontend/conf/logback.xml -Dlogback.debug=false -Dpidfile.path=/dev/null
      - KAFKA_BOOTSTRAP_SERVER=broker:29092
      - DATAHUB_TRACKING_TOPIC=DataHubUsageEvent_v1
      - ELASTIC_CLIENT_HOST=elasticsearch
      - ELASTIC_CLIENT_PORT=9200
#settings for OIDC :
      - AUTH_OIDC_ENABLED=true
      - AUTH_OIDC_CLIENT_ID=***
      - AUTH_OIDC_CLIENT_SECRET=***
      - AUTH_OIDC_DISCOVERY_URI=***
      - AUTH_OIDC_GROUPS_CLAIM=group
      - AUTH_OIDC_EXTRACT_GROUPS_ENABLED=true
      - AUTH_OIDC_JIT_PROVISIONING_ENABLED=true
      - AUTH_OIDC_PRE_PROVISIONING_REQUIRED=false
      - AUTH_JAAS_ENABLED=true
      - AUTH_OIDC_SCOPE=email openid
      - PAC4J_SESSIONSTORE_PROVIDER=PlayCacheSessionStore
      - METADATA_SERVICE_AUTH_ENABLED=true
    hostname: datahub-frontend-react
    image: ${DATAHUB_FRONTEND_IMAGE:-linkedin/datahub-frontend-react}:${DATAHUB_VERSION:-head}
    ports:
      - ${DATAHUB_MAPPED_FRONTEND_PORT:-9002}:9002
    volumes:
      - /home/jjanatka/.datahub/user.props:/etc/datahub/plugins
  datahub-gms:
    container_name: datahub-gms
    depends_on:
      - mysql
    environment:
      - DATAHUB_SERVER_TYPE=${DATAHUB_SERVER_TYPE:-quickstart}
      - DATAHUB_TELEMETRY_ENABLED=${DATAHUB_TELEMETRY_ENABLED:-true}
      - DATAHUB_UPGRADE_HISTORY_KAFKA_CONSUMER_GROUP_ID=generic-duhe-consumer-job-client-gms
      - EBEAN_DATASOURCE_DRIVER=com.mysql.jdbc.Driver
      - EBEAN_DATASOURCE_HOST=mysql:3306
      - EBEAN_DATASOURCE_PASSWORD=datahub
      - EBEAN_DATASOURCE_URL=jdbc:mysql://mysql:3306/datahub?verifyServerCertificate=false&useSSL=true&useUnicode=yes&characterEncoding=UTF-8
      - EBEAN_DATASOURCE_USERNAME=datahub
      - ELASTICSEARCH_HOST=elasticsearch
      - ELASTICSEARCH_INDEX_BUILDER_MAPPINGS_REINDEX=true
      - ELASTICSEARCH_INDEX_BUILDER_SETTINGS_REINDEX=true
      - ELASTICSEARCH_PORT=9200
      - ENTITY_REGISTRY_CONFIG_PATH=/datahub/datahub-gms/resources/entity-registry.yml
      - ENTITY_SERVICE_ENABLE_RETENTION=true
      - ES_BULK_REFRESH_POLICY=WAIT_UNTIL
      - GRAPH_SERVICE_DIFF_MODE_ENABLED=true
      - GRAPH_SERVICE_IMPL=elasticsearch
      - JAVA_OPTS=-Xms1g -Xmx1g
      - KAFKA_BOOTSTRAP_SERVER=broker:29092
      - KAFKA_SCHEMAREGISTRY_URL=http://schema-registry:8081
      - MAE_CONSUMER_ENABLED=true
      - MCE_CONSUMER_ENABLED=true
      - PE_CONSUMER_ENABLED=true
      - UI_INGESTION_ENABLED=true
#auth
      - METADATA_SERVICE_AUTH_ENABLED=true
    hostname: datahub-gms
    image: ${DATAHUB_GMS_IMAGE:-linkedin/datahub-gms}:${DATAHUB_VERSION:-head}
    ports:
      - ${DATAHUB_MAPPED_GMS_PORT:-8080}:8080
    volumes:
      - ${HOME}/.datahub/plugins:/etc/datahub/plugins
  datahub-upgrade:
    command:
      - -u
      - SystemUpdate
    container_name: datahub-upgrade
    environment:
      - EBEAN_DATASOURCE_USERNAME=datahub
      - EBEAN_DATASOURCE_PASSWORD=datahub
      - EBEAN_DATASOURCE_HOST=mysql:3306
      - EBEAN_DATASOURCE_URL=jdbc:mysql://mysql:3306/datahub?verifyServerCertificate=false&useSSL=true&useUnicode=yes&characterEncoding=UTF-8
      - EBEAN_DATASOURCE_DRIVER=com.mysql.jdbc.Driver
      - KAFKA_BOOTSTRAP_SERVER=broker:29092
      - KAFKA_SCHEMAREGISTRY_URL=http://schema-registry:8081
      - ELASTICSEARCH_HOST=elasticsearch
      - ELASTICSEARCH_PORT=9200
      - ELASTICSEARCH_INDEX_BUILDER_MAPPINGS_REINDEX=true
      - ELASTICSEARCH_INDEX_BUILDER_SETTINGS_REINDEX=true
      - ELASTICSEARCH_BUILD_INDICES_CLONE_INDICES=false
      - GRAPH_SERVICE_IMPL=elasticsearch
      - DATAHUB_GMS_HOST=datahub-gms
      - DATAHUB_GMS_PORT=8080
      - ENTITY_REGISTRY_CONFIG_PATH=/datahub/datahub-gms/resources/entity-registry.yml
    hostname: datahub-upgrade
    image: ${DATAHUB_UPGRADE_IMAGE:-acryldata/datahub-upgrade}:${DATAHUB_VERSION:-head}
    labels:
      datahub_setup_job: true
  elasticsearch:
    container_name: elasticsearch
    environment:
      - discovery.type=single-node
      - xpack.security.enabled=false
      - ES_JAVA_OPTS=-Xms256m -Xmx512m -Dlog4j2.formatMsgNoLookups=true
    healthcheck:
      retries: 4
      start_period: 2m
      test:
        - CMD-SHELL
        - curl -sS --fail 'http://localhost:9200/_cluster/health?wait_for_status=yellow&timeout=0s' || exit 1
    hostname: elasticsearch
    image: elasticsearch:7.10.1
    mem_limit: 1g
    ports:
      - ${DATAHUB_MAPPED_ELASTIC_PORT:-9200}:9200
    volumes:
      - esdata:/usr/share/elasticsearch/data
  elasticsearch-setup:
    container_name: elasticsearch-setup
    depends_on:
      - elasticsearch
    environment:
      - ELASTICSEARCH_HOST=elasticsearch
      - ELASTICSEARCH_PORT=9200
      - ELASTICSEARCH_PROTOCOL=http
    hostname: elasticsearch-setup
    image: ${DATAHUB_ELASTIC_SETUP_IMAGE:-linkedin/datahub-elasticsearch-setup}:${DATAHUB_VERSION:-head}
    labels:
      datahub_setup_job: true
  kafka-setup:
    container_name: kafka-setup
    depends_on:
      - broker
      - schema-registry
    environment:
      - DATAHUB_PRECREATE_TOPICS=${DATAHUB_PRECREATE_TOPICS:-false}
      - KAFKA_BOOTSTRAP_SERVER=broker:29092
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
    hostname: kafka-setup
    image: ${DATAHUB_KAFKA_SETUP_IMAGE:-linkedin/datahub-kafka-setup}:${DATAHUB_VERSION:-head}
    labels:
      datahub_setup_job: true
  mysql:
    command: --character-set-server=utf8mb4 --collation-server=utf8mb4_bin --default-authentication-plugin=mysql_native_password
    container_name: mysql
    environment:
      - MYSQL_DATABASE=datahub
      - MYSQL_USER=datahub
      - MYSQL_PASSWORD=datahub
      - MYSQL_ROOT_PASSWORD=datahub
    hostname: mysql
    image: mysql:5.7
    ports:
      - ${DATAHUB_MAPPED_MYSQL_PORT:-3306}:3306
    volumes:
      - ../mysql/init.sql:/docker-entrypoint-initdb.d/init.sql
      - mysqldata:/var/lib/mysql
  mysql-setup:
    labels:
      datahub_setup_job: true
    container_name: mysql-setup
    depends_on:
      - mysql
    environment:
      - MYSQL_HOST=mysql
      - MYSQL_PORT=3306
      - MYSQL_USERNAME=datahub
      - MYSQL_PASSWORD=datahub
      - DATAHUB_DB_NAME=datahub
    hostname: mysql-setup
    image: ${DATAHUB_MYSQL_SETUP_IMAGE:-acryldata/datahub-mysql-setup}:${DATAHUB_VERSION:-head}
  schema-registry:
    container_name: schema-registry
    depends_on:
      - broker
    environment:
      - SCHEMA_REGISTRY_HOST_NAME=schemaregistry
      - SCHEMA_REGISTRY_KAFKASTORE_SECURITY_PROTOCOL=PLAINTEXT
      - SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS=broker:29092
    hostname: schema-registry
    image: confluentinc/cp-schema-registry:7.2.2
    ports:
      - ${DATAHUB_MAPPED_SCHEMA_REGISTRY_PORT:-8081}:8081
  zookeeper:
    container_name: zookeeper
    environment:
      - ZOOKEEPER_CLIENT_PORT=2181
      - ZOOKEEPER_TICK_TIME=2000
    hostname: zookeeper
    image: confluentinc/cp-zookeeper:7.2.2
    ports:
      - ${DATAHUB_MAPPED_ZK_PORT:-2181}:2181
    volumes:
      - zkdata:/var/lib/zookeeper
version: "2.3"
volumes:
  esdata: null
  mysqldata: null
  zkdata: null

cbeauch · 2023-04-05T18:34:39Z

I'm seeing the same thing in my prod env. Reverting back to 0.10.0 for now

cbeauch · 2023-04-05T18:38:44Z

Commenting out PAC4J_SESSIONSTORE_PROVIDER=PlayCacheSessionStore will fix it though

jakobhanna · 2023-04-17T08:09:33Z

Thank you for the workaround, works also for me

github-actions · 2023-05-18T02:05:24Z

This issue is stale because it has been open for 30 days with no activity. If you believe this is still an issue on the latest DataHub release please leave a comment with the version that you tested it with. If this is a question/discussion please head to https://slack.datahubproject.io. For feature requests please use https://feature-requests.datahubproject.io

jakobhanna · 2023-05-18T10:20:50Z

still persists with version 0.10.2

github-actions · 2023-06-18T02:10:13Z

This issue is stale because it has been open for 30 days with no activity. If you believe this is still an issue on the latest DataHub release please leave a comment with the version that you tested it with. If this is a question/discussion please head to https://slack.datahubproject.io. For feature requests please use https://feature-requests.datahubproject.io

github-actions · 2023-07-18T02:22:56Z

This issue was closed because it has been inactive for 30 days since being marked as stale.

jakobhanna added the bug Bug report label Mar 31, 2023

jakobhanna mentioned this issue May 12, 2023

Frontend service couldn't start when set PAC4J_SESSIONSTORE_PROVIDER #7998

Closed

github-actions bot added the stale label May 18, 2023

github-actions bot removed the stale label May 19, 2023

github-actions bot added the stale label Jun 18, 2023

MideO mentioned this issue Jun 30, 2023

fix(datahub-frontend): Add playCaffeine as replacement for removed playEhcache dependency #8344

Merged

5 tasks

github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Jul 18, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OIDC settings in docker-compose file throw error in datahub frontend #7733

OIDC settings in docker-compose file throw error in datahub frontend #7733

jakobhanna commented Mar 31, 2023

jakobhanna commented Mar 31, 2023

cbeauch commented Apr 5, 2023

cbeauch commented Apr 5, 2023

jakobhanna commented Apr 17, 2023

github-actions bot commented May 18, 2023

jakobhanna commented May 18, 2023

github-actions bot commented Jun 18, 2023

github-actions bot commented Jul 18, 2023

OIDC settings in docker-compose file throw error in datahub frontend #7733

OIDC settings in docker-compose file throw error in datahub frontend #7733

Comments

jakobhanna commented Mar 31, 2023

jakobhanna commented Mar 31, 2023

cbeauch commented Apr 5, 2023

cbeauch commented Apr 5, 2023

jakobhanna commented Apr 17, 2023

github-actions bot commented May 18, 2023

jakobhanna commented May 18, 2023

github-actions bot commented Jun 18, 2023

github-actions bot commented Jul 18, 2023