An error occurred when upgrading Keycloak 22.0.5 to 23.0.1

[Dracowyn]

Before reporting an issue

• I have read and understood the above terms for submitting issues, and I understand that my issue may be closed without action if I do not follow them.

Area

core

Describe the bug

Encountered a NullPointerException while performing configuration validation after upgrading to the latest version.

An error occurred when I upgraded the database from Keycloak 22.0.5 to 23.0.1. The version used for the data is MySQL 8, and the encoding is utf8mb4_unicode_ci.

The following is the detailed error message:

2023-11-30 22:56:15,645 INFO [org.infinispan.CONTAINER] (keycloak-cache-init) ISPN000556: Starting user marshaller 'org.infinispan.jboss.marshalling.core.JBossUserMarshaller'
2023-11-30 22:56:19,608 INFO [org.keycloak.quarkus.runtime.storage.legacy.liquibase.QuarkusJpaUpdaterProvider] (main) Updating database. Using changelog META-INF/jpa-changelog-master.xml
2023-11-30 22:56:24,789 WARN [liquibase.executor.jvm.JdbcExecutor] (main) 'utf8' is currently an alias for the character set UTF8MB3, but will be an alias for UTF8MB4 in a future release. Please consider using UTF8MB4 in order to be unambiguous.
2023-11-30 22:56:24,908 WARN [liquibase.executor.jvm.JdbcExecutor] (main) 'utf8' is currently an alias for the character set UTF8MB3, but will be an alias for UTF8MB4 in a future release. Please consider using UTF8MB4 in order to be unambiguous.
2023-11-30 22:56:24,946 WARN [liquibase.executor.jvm.JdbcExecutor] (main) 'utf8' is currently an alias for the character set UTF8MB3, but will be an alias for UTF8MB4 in a future release. Please consider using UTF8MB4 in order to be unambiguous.

UPDATE SUMMARY
Run: 4
Previously run: 113
Filtered out: 0

Total change sets: 117

2023-11-30 22:56:24,967 INFO [cc.coopersoft.keycloak.phone.providers.jpa.TokenCodeJpaEntityProviderFactory] (main) Create TokenCodeJpaEntityProvider
2023-11-30 22:56:25,487 INFO [org.keycloak.connections.infinispan.DefaultInfinispanConnectionProviderFactory] (main) Node name: node_470475, Site name: null
2023-11-30 22:56:26,929 WARN [io.agroal.pool] (main) Datasource '': JDBC resources leaked: 1 ResultSet(s) and 1 Statement(s)
2023-11-30 22:56:27,081 ERROR [org.keycloak.quarkus.runtime.cli.ExecutionExceptionHandler] (main) ERROR: Failed to start server in (development) mode
2023-11-30 22:56:27,083 ERROR [org.keycloak.quarkus.runtime.cli.ExecutionExceptionHandler] (main) Error details:: java.lang.NullPointerException: Cannot invoke "org.keycloak.models.RealmModel.getClientScopesStream()" because "realm" is null
at org.keycloak.userprofile.config.UPConfigUtils.validateScopes(UPConfigUtils.java:195)
at org.keycloak.userprofile.config.UPConfigUtils.validateAttribute(UPConfigUtils.java:165)
at org.keycloak.userprofile.config.UPConfigUtils.lambda$validateAttributes$2(UPConfigUtils.java:118)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at org.keycloak.userprofile.config.UPConfigUtils.validateAttributes(UPConfigUtils.java:118)
at org.keycloak.userprofile.config.UPConfigUtils.validate(UPConfigUtils.java:95)
at org.keycloak.userprofile.DeclarativeUserProfileProvider.validateConfiguration(DeclarativeUserProfileProvider.java:199)
at org.keycloak.models.jpa.RealmAdapter.updateComponent(RealmAdapter.java:2088)
at org.keycloak.migration.migrators.MigrateTo23_0_0.updateUserProfileConfig(MigrateTo23_0_0.java:82)
at org.keycloak.migration.migrators.MigrateTo23_0_0.migrateRealm(MigrateTo23_0_0.java:55)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
at org.keycloak.utils.ClosingStream.forEach(ClosingStream.java:128)
at org.keycloak.migration.migrators.MigrateTo23_0_0.migrate(MigrateTo23_0_0.java:46)
at org.keycloak.storage.datastore.LegacyMigrationManager.migrate(LegacyMigrationManager.java:140)
at org.keycloak.migration.MigrationModelManager.migrate(MigrationModelManager.java:33)
at org.keycloak.quarkus.runtime.storage.legacy.database.LegacyJpaConnectionProviderFactory.migrateModel(LegacyJpaConnectionProviderFactory.java:216)
at org.keycloak.quarkus.runtime.storage.legacy.database.LegacyJpaConnectionProviderFactory.initSchema(LegacyJpaConnectionProviderFactory.java:210)
at org.keycloak.models.utils.KeycloakModelUtils.lambda$runJobInTransaction$1(KeycloakModelUtils.java:260)
at org.keycloak.models.utils.KeycloakModelUtils.runJobInTransactionWithResult(KeycloakModelUtils.java:382)
at org.keycloak.models.utils.KeycloakModelUtils.runJobInTransaction(KeycloakModelUtils.java:259)
at org.keycloak.quarkus.runtime.storage.legacy.database.LegacyJpaConnectionProviderFactory.postInit(LegacyJpaConnectionProviderFactory.java:135)
at org.keycloak.quarkus.runtime.integration.QuarkusKeycloakSessionFactory.init(QuarkusKeycloakSessionFactory.java:105)
at org.keycloak.quarkus.runtime.integration.jaxrs.QuarkusKeycloakApplication.createSessionFactory(QuarkusKeycloakApplication.java:56)
at org.keycloak.services.resources.KeycloakApplication.startup(KeycloakApplication.java:124)
at org.keycloak.quarkus.runtime.integration.jaxrs.QuarkusKeycloakApplication.onStartupEvent(QuarkusKeycloakApplication.java:46)
at org.keycloak.quarkus.runtime.integration.jaxrs.QuarkusKeycloakApplication_Observer_onStartupEvent_67d48587b481b764f44181a34540ebd3d495c2c7.notify(Unknown Source)
at io.quarkus.arc.impl.EventImpl$Notifier.notifyObservers(EventImpl.java:346)
at io.quarkus.arc.impl.EventImpl$Notifier.notify(EventImpl.java:328)
at io.quarkus.arc.impl.EventImpl.fire(EventImpl.java:82)
at io.quarkus.arc.runtime.ArcRecorder.fireLifecycleEvent(ArcRecorder.java:155)
at io.quarkus.arc.runtime.ArcRecorder.handleLifecycleEvents(ArcRecorder.java:106)
at io.quarkus.deployment.steps.LifecycleEventsBuildStep$startupEvent1144526294.deploy_0(Unknown Source)
at io.quarkus.deployment.steps.LifecycleEventsBuildStep$startupEvent1144526294.deploy(Unknown Source)
at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
at io.quarkus.runtime.Application.start(Application.java:101)
at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:111)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:71)
at org.keycloak.quarkus.runtime.KeycloakMain.start(KeycloakMain.java:117)
at org.keycloak.quarkus.runtime.cli.command.AbstractStartCommand.run(AbstractStartCommand.java:33)
at picocli.CommandLine.executeUserObject(CommandLine.java:2026)
at picocli.CommandLine.access$1500(CommandLine.java:148)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2453)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2415)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273)
at picocli.CommandLine$RunLast.execute(CommandLine.java:2417)
at picocli.CommandLine.execute(CommandLine.java:2170)
at org.keycloak.quarkus.runtime.cli.Picocli.parseAndRun(Picocli.java:119)
at org.keycloak.quarkus.runtime.KeycloakMain.main(KeycloakMain.java:107)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at io.quarkus.bootstrap.runner.QuarkusEntryPoint.doRun(QuarkusEntryPoint.java:61)
at io.quarkus.bootstrap.runner.QuarkusEntryPoint.main(QuarkusEntryPoint.java:32)

Version

23.0.1

Expected behavior

The database structure is upgraded normally and Keycloak is initialized normally.

Actual behavior

ERROR [org.keycloak.quarkus.runtime.cli.ExecutionExceptionHandler] (main) Error details:: java.lang.NullPointerException: Cannot invoke "org.keycloak.models.RealmModel.getClientScopesStream()" because "realm" is null

How to Reproduce?

This problem will occur when Keycloak 22~23 versions that already have certain data are upgraded to 24, and Java17+MySQL8 is used.

Anything else?

No response

[lanmark7]

This is what I am getting on multiple environments that happened to be set to keycloak:latest and automatically upgraded. They now all throw that "because 'realm' is null" error. My configurations is Azure App Service running the keycloak container with a Postgres database.

I have resorted to restoring old databases and setting the container to 22.0 tag.

[DavidHaltinner]

EDIT: Now that I read things again, I am getting:
Cannot invoke "org.keycloak.models.RealmModel.getMasterAdminClient()" because "realm" is null
in the logs, where the original post is coming from:
Cannot invoke "org.keycloak.models.RealmModel.getClientScopesStream()" because "realm" is null

So mine from getMasterAdminClient, the original post from getClientScopesStream, but same error. Maybe they are not related after all, likely best I just do a new issue for mine i think.

[lanmark7]

EDIT: Now that I read things again, I am getting: Cannot invoke "org.keycloak.models.RealmModel.getMasterAdminClient()" because "realm" is null in the logs, where the original post is coming from: Cannot invoke "org.keycloak.models.RealmModel.getClientScopesStream()" because "realm" is null

So mine from getMasterAdminClient, the original post from getClientScopesStream, but same error. Maybe they are not related after all, likely best I just do a new issue for mine i think.

In mine it won't even start up the Keycloak container and dies. I am able to change the version back to 22.0.5 and it does boot so it seems that it never gets to updating the database schema.

[NikolaBerovic]

The issue has been fixed:
#25325

A workaround: use the latest nighlty version for migration process to start, then switch to 23.0.1, it works properly.
Anyways I would wait for 23.0.2 since there are couple of other issues in 23.0.1 (like when creating user and canceling it, it results in error, this has been fixed also, and will be featured in the next version)

[sschu]

Thanks @NikolaBerovic . Closing this as a duplicate of #25183.

[aglv]

I also ran into this problem. Thanks to this thread, I fixed it by replacing my

command:
  - "binary-name"
  - "option1=value1"

to

command: [ "binary-name",
  "option1", "value1" ]

This works with v23.0.7