-
Notifications
You must be signed in to change notification settings - Fork 6.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Keycloak 17 installation broken #11316
Comments
Downgrading zlib (1:1.2.12-1 => 1:1.2.11-5) worked to get around the issue. |
Now it strands at this point kc.sh --verbose -cf /etc/keycloak/keycloak.conf start
2022-04-14 18:42:43,969 INFO [org.keycloak.quarkus.runtime.hostname.DefaultHostnameProvider] (main) Hostname settings: FrontEnd: login.systemat.is, Strict HTTPS: true, Path: <request>, Strict BackChannel: false, Admin: <request>, Port: 8,087, Proxied: true
2022-04-14 18:42:45,333 WARN [org.infinispan.PERSISTENCE] (keycloak-cache-init) ISPN000554: jboss-marshalling is deprecated and planned for removal
2022-04-14 18:42:45,409 WARN [org.infinispan.CONFIG] (keycloak-cache-init) ISPN000569: Unable to persist Infinispan internal caches as no global state enabled
2022-04-14 18:42:45,439 INFO [org.infinispan.CONTAINER] (keycloak-cache-init) ISPN000556: Starting user marshaller 'org.infinispan.jboss.marshalling.core.JBossUserMarshaller'
2022-04-14 18:42:45,707 INFO [org.infinispan.CONTAINER] (keycloak-cache-init) ISPN000128: Infinispan version: Infinispan 'Triskaidekaphobia' 13.0.6.Final
2022-04-14 18:42:45,822 INFO [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000078: Starting JGroups channel `ISPN`
2022-04-14 18:42:45,823 INFO [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000088: Unable to use any JGroups configuration mechanisms provided in properties {}. Using default JGroups configuration!
2022-04-14 18:42:45,918 WARN [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the send buffer of socket MulticastSocket was set to 1.00MB, but the OS only allocated 212.99KB
2022-04-14 18:42:45,919 WARN [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the receive buffer of socket MulticastSocket was set to 20.00MB, but the OS only allocated 212.99KB
2022-04-14 18:42:45,920 WARN [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the send buffer of socket MulticastSocket was set to 1.00MB, but the OS only allocated 212.99KB
2022-04-14 18:42:45,921 WARN [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the receive buffer of socket MulticastSocket was set to 25.00MB, but the OS only allocated 212.99KB
2022-04-14 18:42:47,933 INFO [org.jgroups.protocols.pbcast.GMS] (keycloak-cache-init) gamla-50976: no members discovered after 2002 ms: creating cluster as coordinator
2022-04-14 18:42:47,945 INFO [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000094: Received new cluster view for channel ISPN: [gamla-50976|0] (1) [gamla-50976]
2022-04-14 18:42:47,951 INFO [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000079: Channel `ISPN` local address is `gamla-50976`, physical addresses are `[192.168.3.100:46092]`
2022-04-14 18:42:48,572 INFO [org.keycloak.connections.infinispan.DefaultInfinispanConnectionProviderFactory] (main) Node name: gamla-50976, Site name: null
2022-04-14 18:42:50,540 INFO [org.infinispan.CLUSTER] (main) ISPN000080: Disconnecting JGroups channel `ISPN`
2022-04-14 18:42:50,618 ERROR [org.keycloak.quarkus.runtime.cli.ExecutionExceptionHandler] (main) ERROR: Failed to start server in (production) mode
2022-04-14 18:42:50,618 ERROR [org.keycloak.quarkus.runtime.cli.ExecutionExceptionHandler] (main) Error details:: java.lang.NullPointerException
at org.keycloak.models.cache.infinispan.entities.CachedRealm.<init>(CachedRealm.java:266)
at org.keycloak.models.cache.infinispan.RealmCacheSession.getRealm(RealmCacheSession.java:414)
at org.keycloak.services.managers.ApplianceBootstrap.isNewInstall(ApplianceBootstrap.java:46)
at org.keycloak.quarkus.runtime.storage.database.jpa.QuarkusJpaConnectionProviderFactory.createMasterRealm(QuarkusJpaConnectionProviderFactory.java:232)
at org.keycloak.quarkus.runtime.storage.database.jpa.QuarkusJpaConnectionProviderFactory.initSchema(QuarkusJpaConnectionProviderFactory.java:199)
at org.keycloak.models.utils.KeycloakModelUtils.runJobInTransaction(KeycloakModelUtils.java:239)
at org.keycloak.quarkus.runtime.storage.database.jpa.QuarkusJpaConnectionProviderFactory.postInit(QuarkusJpaConnectionProviderFactory.java:136)
at org.keycloak.quarkus.runtime.integration.QuarkusKeycloakSessionFactory.init(QuarkusKeycloakSessionFactory.java:96)
at org.keycloak.quarkus.runtime.integration.jaxrs.QuarkusKeycloakApplication.initializeKeycloakSessionFactory(QuarkusKeycloakApplication.java:68)
at org.keycloak.quarkus.runtime.integration.jaxrs.QuarkusKeycloakApplication.startup(QuarkusKeycloakApplication.java:49)
at org.keycloak.quarkus.runtime.integration.QuarkusLifecycleObserver.onStartupEvent(QuarkusLifecycleObserver.java:37)
at org.keycloak.quarkus.runtime.integration.QuarkusLifecycleObserver_Observer_onStartupEvent_b0e82415b143738dc1f986a5fa4668e83d0a5dea.notify(Unknown Source)
at io.quarkus.arc.impl.EventImpl$Notifier.notifyObservers(EventImpl.java:320)
at io.quarkus.arc.impl.EventImpl$Notifier.notify(EventImpl.java:302)
at io.quarkus.arc.impl.EventImpl.fire(EventImpl.java:73)
at io.quarkus.arc.runtime.ArcRecorder.fireLifecycleEvent(ArcRecorder.java:128)
at io.quarkus.arc.runtime.ArcRecorder.handleLifecycleEvents(ArcRecorder.java:97)
at io.quarkus.deployment.steps.LifecycleEventsBuildStep$startupEvent1144526294.deploy_0(Unknown Source)
at io.quarkus.deployment.steps.LifecycleEventsBuildStep$startupEvent1144526294.deploy(Unknown Source)
at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
at io.quarkus.runtime.Application.start(Application.java:101)
at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:103)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:67)
at org.keycloak.quarkus.runtime.KeycloakMain.start(KeycloakMain.java:86)
at org.keycloak.quarkus.runtime.cli.command.AbstractStartCommand.run(AbstractStartCommand.java:34)
at picocli.CommandLine.executeUserObject(CommandLine.java:1939)
at picocli.CommandLine.access$1300(CommandLine.java:145)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2358)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2352)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2314)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
at picocli.CommandLine$RunLast.execute(CommandLine.java:2316)
at picocli.CommandLine.execute(CommandLine.java:2078)
at org.keycloak.quarkus.runtime.cli.Picocli.parseAndRun(Picocli.java:84)
at org.keycloak.quarkus.runtime.KeycloakMain.main(KeycloakMain.java:77)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at io.quarkus.bootstrap.runner.QuarkusEntryPoint.doRun(QuarkusEntryPoint.java:60)
at io.quarkus.bootstrap.runner.QuarkusEntryPoint.main(QuarkusEntryPoint.java:31)
|
I couldn't directly use the existing database. I had to let keycloak 17 create a new database and then swap them after that. Then it started up. |
@EinarArnason Could you please describe the steps you did during the upgrade? |
|
Just ran into this issue: NixOS/nixpkgs#170539 — we have a zlib ≥ 1.2.12 on NixOS too, I performed the same operations as @EinarArnason Anyone understand how Quarkus works? |
@RaitoBezarius The code failing is not really related to Quarkus but to Keycloak itself. The file failing the CRC check is indeed generated by Quarkus (lib/quarkus/generated-bytecode.jar) during re-augmentation but we are reading it in Keycloak to load the properties set when running the build command. I'm not sure why CRC is failing for that file but if changing zlib version solves the problem, perhaps this can be a bug in the JVM version? Or even with zlib? |
@pedroigor it is a bit of a shame to have to downgrade zlib, how can we debug further the issue? |
@RaitoBezarius Not really in favor of the downgrade. I'm kinda intrigued about this issue and why the JAR is corrupted. I would like to reproduce it locally but I did manage to do it. Do you have the steps I can follow to reproduce it? |
@pedroigor If you have Nix in the hand: It seems like @EinarArnason succeeded to get this issue on Archlinux with a recent enough zlib, I assume it requires also a configuration file. In the original NixOS automatic test, we tested with providing a snakeoil SSL certificate, and it passed, if I remove these, it fails. To be precise, here's our build phase: https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/servers/keycloak/default.nix#L25-L46 — the configuration we install is https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/servers/keycloak/default.nix#L28 ; then we install plugins if any and run the build. We crash there: https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/servers/keycloak/default.nix#L43 So to summary:
I mailed the zlib authors to ask if they had any idea on what could be going on. |
As per NixOS/nixpkgs#177053 We have found out the issue is related to a change in how zlib verifies checksums, see madler/zlib#618 and a patch madler/zlib@ec3df00 seems to fix the issue. It is not specific to Keycloak except that Keycloak uses Quarkus augmentation feature, which seems to be reliant on zlib or some CRC implementations that I am not aware of. If Keycloak people are fine with it, I'm okay with closing the issue here, @pedroigor — at some point, upstream will release a new version and rolling release distributions will catch up + backport and other distributions will skip this version. |
@RaitoBezarius Thanks, I appreciate your help and feedback on this one. I'll close the issue then. Looks like other projects, like Flyway, are also having a similar issue as per this link madler/zlib#618. So I guess this "bad behavior" now being reverted is JDK-related and not specific to a project. If you think it makes sense I can reach out Quarkus team (as there are more smart people than me there to look at this). |
@pedroigor If you have an easier way to give a heads-up to the Quarkus team, that would be great, I think. Of course, I think we tried hard enough to link all different threads to let everyone who run into this figure out what to do, so they might stumble on this by themselves I imagine. Thank you for your time and quick answers! |
Starting zlib 1.2.12, CRC validation has became stricter. This broke Keycloak ≥ 17 in certain situations, for details, see: - keycloak/keycloak#11316 ; - NixOS#170539 This patch makes the CRC validation comprehensive with respect to older or already existing checksums out there.
Starting zlib 1.2.12, CRC validation has became stricter. This broke Keycloak ≥ 17 in certain situations, for details, see: - keycloak/keycloak#11316 ; - #170539 This patch makes the CRC validation comprehensive with respect to older or already existing checksums out there. (cherry picked from commit 8335c46)
Starting zlib 1.2.12, CRC validation has became stricter. This broke Keycloak ≥ 17 in certain situations, for details, see: - keycloak/keycloak#11316 ; - #170539 This patch makes the CRC validation comprehensive with respect to older or already existing checksums out there. (cherry picked from commit 8335c46)
@RaitoBezarius @EinarArnason Tky u. Created quarkusio/quarkus#26046. |
Starting zlib 1.2.12, CRC validation has became stricter. This broke Keycloak ≥ 17 in certain situations, for details, see: - keycloak/keycloak#11316 ; - NixOS#170539 This patch makes the CRC validation comprehensive with respect to older or already existing checksums out there.
Starting zlib 1.2.12, CRC validation has became stricter. This broke Keycloak ≥ 17 in certain situations, for details, see: - keycloak/keycloak#11316 ; - NixOS#170539 This patch makes the CRC validation comprehensive with respect to older or already existing checksums out there.
Starting zlib 1.2.12, CRC validation has became stricter. This broke Keycloak ≥ 17 in certain situations, for details, see: - keycloak/keycloak#11316 ; - NixOS#170539 This patch makes the CRC validation comprehensive with respect to older or already existing checksums out there.
Starting zlib 1.2.12, CRC validation has became stricter. This broke Keycloak ≥ 17 in certain situations, for details, see: - keycloak/keycloak#11316 ; - NixOS#170539 This patch makes the CRC validation comprehensive with respect to older or already existing checksums out there.
Starting zlib 1.2.12, CRC validation has became stricter. This broke Keycloak ≥ 17 in certain situations, for details, see: - keycloak/keycloak#11316 ; - NixOS#170539 This patch makes the CRC validation comprehensive with respect to older or already existing checksums out there.
Starting zlib 1.2.12, CRC validation has became stricter. This broke Keycloak ≥ 17 in certain situations, for details, see: - keycloak/keycloak#11316 ; - NixOS#170539 This patch makes the CRC validation comprehensive with respect to older or already existing checksums out there.
Starting zlib 1.2.12, CRC validation has became stricter. This broke Keycloak ≥ 17 in certain situations, for details, see: - keycloak/keycloak#11316 ; - NixOS#170539 This patch makes the CRC validation comprehensive with respect to older or already existing checksums out there.
Starting zlib 1.2.12, CRC validation has became stricter. This broke Keycloak ≥ 17 in certain situations, for details, see: - keycloak/keycloak#11316 ; - NixOS#170539 This patch makes the CRC validation comprehensive with respect to older or already existing checksums out there.
Starting zlib 1.2.12, CRC validation has became stricter. This broke Keycloak ≥ 17 in certain situations, for details, see: - keycloak/keycloak#11316 ; - NixOS#170539 This patch makes the CRC validation comprehensive with respect to older or already existing checksums out there.
Describe the bug
After upgrading on Arch Linux from 16.1.0 to 17.x I can't get past this error:
Version
17.0.1
Expected behavior
No response
Actual behavior
No response
How to Reproduce?
No response
Anything else?
No response
The text was updated successfully, but these errors were encountered: