Keycloak fails to start when uninstalling custom provider #24459
Comments
ghost
added
|
Maybe, it should be a good idea to include a kind of management page for these things... |
|
Have you executed the build step as mentioned in the documentation? Like: 4. Stop the server 5. run build 6. Start server? |
|
Yeah. Both cases: Executing it and without executing it. No one worked. |
|
And you are not doing a rolling restart where another pod is still present but you completely shut down all running instances of Keycloak? |
|
No more pods present. |
|
I'm facing the same issue. Removing any Provider jar from the Removing Providers from the |
|
Is it happening in the latest version? Wasn't able to reproduce in 22.0.5, 23.0.3, and nightly. I've seen a similar issue before and I think we have an open issue for it. @vmuzikar Wdyt? |
pedroigor
added
team/cloud-native
area/dist/quarkus
and removed
team/core
area/core
labels
|
I just tested this again on 23.0.3. I removed a provider jar file from the |
@pedroigor Seems to be affecting only some providers? |
|
As a test to check this behaviour, let me suggestto create a mapper, create the Jar file, put it in the "providers" folder, edit the project (changing anything) and put the new created Jar file into the folder and try to start-up Keycloak again and check. |
|
@PIPOGit @dpulrichth I saw similar behavior to @pedroigor with 23.0.4 - build, start, or state-dev all appropriately detected the provider jar removal. However on linux at least there was no logging of the exception shown by @dpulrichth - not to the console by default, nor at the debug level. This was with both the rest example provider and the mapper provider referenced by @PIPOGit above. @PIPOGit can you retry your scenario on a later release to confirm if this is still an issue for you? |
|
Hi, @shawkins. I hope You can "read" this PowerShell script... and it works! (adapt it as you wish) I have TWO questions:
The script with my steps. # ================================================================
# Launch a PowerShell console
# CONSOLE["Test KeyCloak Console"]
# Configure console.
Clear-Host;
# Console's title
$Host.UI.RawUI.WindowTitle = "Test KeyCloak Console";
# Some variables
$ROOT_FOLDER = "C:\Temp\Test KeyCloak";
$KEYCLOAK_VERSION = "23.0.6";
$KEYCLOAK_HOME = "keycloak-$($KEYCLOAK_VERSION)";
# Operations
Set-Location "${ROOT_FOLDER}"
# Download KeyCloak "latest" version
# Check: https://github.com/keycloak/keycloak/releases/
# if ( !( Test-Path ${ROOT_FOLDER} -PathType Container ) ) {
# New-Item -ItemType Directory -Force -Path ${ROOT_FOLDER} | Out-Null;
# }
# Download file
Set-Location "${ROOT_FOLDER}"
$ProgressPreference = 'SilentlyContinue' # Subsequent calls do not display UI.
IWR "https://github.com/keycloak/keycloak/releases/download/$($KEYCLOAK_VERSION)/$($KEYCLOAK_HOME).zip" -OutFile "$($KEYCLOAK_HOME).zip"
$ProgressPreference = 'Continue' # Subsequent calls do display UI.
# FILE: [${ROOT_FOLDER}\$($KEYCLOAK_HOME).zip] (164.677.937 bytes)
# Unzip downloaded file
$ProgressPreference = 'SilentlyContinue' # Subsequent calls do not display UI.
Expand-Archive -Path "${ROOT_FOLDER}\$($KEYCLOAK_HOME).zip" -DestinationPath "${ROOT_FOLDER}\KeyCloak"
$ProgressPreference = 'Continue' # Subsequent calls do display UI.
# FOLDER: [${ROOT_FOLDER}\KeyCloak\$($KEYCLOAK_HOME)]
# ================================================================
# Start KeyCloak
# Launch a new PowerShell console
# CONSOLE["Test KeyCloak Server"]
Clear-Host;
# Console's title
$Host.UI.RawUI.WindowTitle = "Test KeyCloak Server";
# Some variables
$ROOT_FOLDER = "C:\Temp\Test KeyCloak";
$KEYCLOAK_VERSION = "23.0.6";
$KEYCLOAK_HOME = "keycloak-$($KEYCLOAK_VERSION)";
# Operations
Set-Location "${ROOT_FOLDER}"
# Start server
Set-Location "${ROOT_FOLDER}\KeyCloak\$($KEYCLOAK_HOME)\bin"
& .\kc.bat start-dev
# Stop server with ^C.
# (will run after ^C)
Set-Location "${ROOT_FOLDER}"
# ================================================================
# ================================================================
# Open Admin console at: http://localhost:8080/
# CONSOLE["Test KeyCloak Console"]
Clear-Host;
Start-Process "http://localhost:8080/";
# CHECK it works.
# CREATE [admin/admin] account
# LOGIN
# CHECK it works.
# STOP the server in server's console
# ================================================================
# ================================================================
# Create mapper: "Mapper 01"
# CONSOLE["Test KeyCloak Console"]
# Use: https://betterjavacode.com/programming/custom-protocol-mapper-keycloak
# Mapper 01
Set-Location "${ROOT_FOLDER}\Custom Mappers\mapper-01"
Clear-Host;
# Maven: Create custom mapper file
mvn clean package;
# Copy custom mapper file
Copy-Item -Path "${ROOT_FOLDER}\Custom Mappers\mapper-01\target\pipogit-cpm-01.jar" -Destination "${ROOT_FOLDER}\KeyCloak\$($KEYCLOAK_HOME)\providers";
# List custom mappers
Get-ChildItem "${ROOT_FOLDER}\KeyCloak\$($KEYCLOAK_HOME)\providers\*.jar";
# CHECK "pipogit-cpm-01.jar" is listed.
# Back to HOME
Set-Location "${ROOT_FOLDER}"
# ================================================================
# ================================================================
# Launch KeyCloak Server again
# CONSOLE["Test KeyCloak Server"]
# Repeat previous steps
Clear-Host;
# Operations
Set-Location "${ROOT_FOLDER}"
# Start server
Set-Location "${ROOT_FOLDER}\KeyCloak\$($KEYCLOAK_HOME)\bin"
& .\kc.bat start-dev
# Stop server with ^C.
# (will run after ^C)
Set-Location "${ROOT_FOLDER}"
# ================================================================
# ================================================================
# Open Admin console at: http://localhost:8080/
# CONSOLE["Test KeyCloak Console"]
Clear-Host;
Start-Process "http://localhost:8080/";
# Login
# GOTO: Client Scopes
# Select one (i.e.: "address")
# GOTO: "Mappers"
# Select: "Add mapper" > "By configuration"
# CHECK that "PipoGIT Custom Mapper 01" exists.
# STOP the server in server's console
# ================================================================
# ================================================================
# Create mapper: "Mapper 02"
# CONSOLE["Test KeyCloak Console"]
# Use: https://betterjavacode.com/programming/custom-protocol-mapper-keycloak
# Mapper 02
Set-Location "${ROOT_FOLDER}\Custom Mappers\mapper-02"
Clear-Host;
# Maven: Create custom mapper file
mvn clean package;
# Copy custom mapper file
Copy-Item -Path "${ROOT_FOLDER}\Custom Mappers\mapper-02\target\pipogit-cpm-02.jar" -Destination "${ROOT_FOLDER}\KeyCloak\$($KEYCLOAK_HOME)\providers";
# List custom mappers
Get-ChildItem "${ROOT_FOLDER}\KeyCloak\$($KEYCLOAK_HOME)\providers\*.jar";
# CHECK "pipogit-cpm-01.jar" AND "pipogit-cpm-02.jar" are listed.
# Back to HOME
Set-Location "${ROOT_FOLDER}"
# ================================================================
# ================================================================
# Launch KeyCloak Server again
# CONSOLE["Test KeyCloak Server"]
# Repeat previous steps
Clear-Host;
# Operations
Set-Location "${ROOT_FOLDER}"
# Start server
Set-Location "${ROOT_FOLDER}\KeyCloak\$($KEYCLOAK_HOME)\bin"
& .\kc.bat start-dev
# Stop server with ^C.
Set-Location "${ROOT_FOLDER}"
# ================================================================
# ================================================================
# Open Admin console at: http://localhost:8080/
# CONSOLE["Test KeyCloak Console"]
Clear-Host;
Start-Process "http://localhost:8080/";
# Login
# GOTO: Client Scopes
# Select one (i.e.: "address")
# GOTO: "Mappers"
# Select: "Add mapper" > "By configuration"
# CHECK that "PipoGIT Custom Mapper 01" exists.
# CHECK that "PipoGIT Custom Mapper 02" DOES NOT exists.
# STOP the server in server's console
# ================================================================
# ================================================================
#
# *** FINAL TEST ***
#
# CONSOLE["Test KeyCloak Console"]
Clear-Host;
# List custom mappers
Get-ChildItem "${ROOT_FOLDER}\KeyCloak\$($KEYCLOAK_HOME)\providers\*.jar";
# Delete "Custom Mapper 02" from "pipogit-cpm-02.jar"
Remove-Item "${ROOT_FOLDER}\KeyCloak\$($KEYCLOAK_HOME)\providers\pipogit-cpm-02.jar";
# List custom mappers
Get-ChildItem "${ROOT_FOLDER}\KeyCloak\$($KEYCLOAK_HOME)\providers\*.jar";
# ================================================================
# ================================================================
# Launch KeyCloak Server again
# CONSOLE["Test KeyCloak Server"]
# Repeat previous steps
Clear-Host;
# Operations
Set-Location "${ROOT_FOLDER}"
# Start server
Set-Location "${ROOT_FOLDER}\KeyCloak\$($KEYCLOAK_HOME)\bin"
& .\kc.bat start-dev
# CHECK Error Log Messages:
<#
The DelayedHandler was closed before any children handlers were configured. Messages will be written to stderr.
2024-02-15 08:44:36,341 DEBUG [org.jboss.logging] (main) Logging Provider: org.jboss.logging.JBossLogManagerProvider
Exception in thread "main" java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:119)
at java.base/java.lang.reflect.Method.invoke(Method.java:578)
at io.quarkus.bootstrap.runner.QuarkusEntryPoint.doRun(QuarkusEntryPoint.java:61)
at io.quarkus.bootstrap.runner.QuarkusEntryPoint.main(QuarkusEntryPoint.java:32)
Caused by: java.lang.RuntimeException: Failed to open C:\Temp\Test KeyCloak\KeyCloak\keycloak-23.0.6\lib\..\providers\pipogit-cpm-02.jar
at io.quarkus.bootstrap.runner.JarResource.ensureJarFileIsOpen(JarResource.java:172)
at io.quarkus.bootstrap.runner.JarResource.readLockAcquireAndGetJarReference(JarResource.java:153)
at io.quarkus.bootstrap.runner.JarResource.getResourceURL(JarResource.java:101)
at io.quarkus.bootstrap.runner.RunnerClassLoader.findResources(RunnerClassLoader.java:214)
at java.base/java.lang.ClassLoader.getResources(ClassLoader.java:1469)
at io.smallrye.common.classloader.ClassPathUtils.consumeAsPaths(ClassPathUtils.java:84)
at io.smallrye.config.AbstractLocationConfigSourceLoader.tryClassPath(AbstractLocationConfigSourceLoader.java:139)
at io.smallrye.config.AbstractLocationConfigSourceLoader.loadConfigSources(AbstractLocationConfigSourceLoader.java:102)
at io.smallrye.config.AbstractLocationConfigSourceLoader.loadConfigSources(AbstractLocationConfigSourceLoader.java:85)
at io.quarkus.runtime.configuration.ApplicationPropertiesConfigSourceLoader$InClassPath.getConfigSources(ApplicationPropertiesConfigSourceLoader.java:30)
at io.quarkus.runtime.configuration.ApplicationPropertiesConfigSourceLoader$InClassPath.getConfigSources(ApplicationPropertiesConfigSourceLoader.java:27)
at io.smallrye.config.SmallRyeConfig$ConfigSources.buildSources(SmallRyeConfig.java:581)
at io.smallrye.config.SmallRyeConfig$ConfigSources.<init>(SmallRyeConfig.java:537)
at io.smallrye.config.SmallRyeConfig.<init>(SmallRyeConfig.java:69)
at io.smallrye.config.SmallRyeConfigBuilder.build(SmallRyeConfigBuilder.java:629)
at io.quarkus.runtime.configuration.QuarkusConfigFactory.getConfigFor(QuarkusConfigFactory.java:29)
at io.smallrye.config.SmallRyeConfigProviderResolver.getConfig(SmallRyeConfigProviderResolver.java:76)
at io.smallrye.config.SmallRyeConfigProviderResolver.getConfig(SmallRyeConfigProviderResolver.java:64)
at org.keycloak.quarkus.runtime.configuration.Configuration.getConfig(Configuration.java:51)
at org.keycloak.quarkus.runtime.cli.Picocli.hasConfigChanges(Picocli.java:335)
at org.keycloak.quarkus.runtime.cli.Picocli.requiresReAugmentation(Picocli.java:169)
at org.keycloak.quarkus.runtime.cli.Picocli.runReAugmentationIfNeeded(Picocli.java:153)
at org.keycloak.quarkus.runtime.cli.Picocli.parseAndRun(Picocli.java:105)
at org.keycloak.quarkus.runtime.KeycloakMain.main(KeycloakMain.java:107)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
... 3 more
Caused by: java.nio.file.NoSuchFileException: C:\Temp\Test KeyCloak\KeyCloak\keycloak-23.0.6\lib\..\providers\pipogit-cpm-02.jar
at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:85)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
at java.base/sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
at java.base/sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
at java.base/sun.nio.fs.WindowsFileSystemProvider.readAttributes(WindowsFileSystemProvider.java:197)
at java.base/java.nio.file.Files.readAttributes(Files.java:1848)
at java.base/java.util.zip.ZipFile$Source.get(ZipFile.java:1414)
at java.base/java.util.zip.ZipFile$CleanableResource.<init>(ZipFile.java:723)
at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:250)
at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:179)
at java.base/java.util.jar.JarFile.<init>(JarFile.java:339)
at io.smallrye.common.io.jar.JarFiles.create(JarFiles.java:33)
at io.quarkus.bootstrap.runner.JarResource.ensureJarFileIsOpen(JarResource.java:170)
... 27 more
#>
# ================================================================
|
The issue is with including the application.properties file in your provider jar. We are first checking for config changes, then for provider changes. To resolve the config, we try to load all of the known application.properties files - thus the exception here. If you omit that file, then things will work as expected.
I see a split package warning on the cli related to this. What behavior would you like to see? @dpulrichth stack also seems related to, and potentially resolved by #19285 @vmuzikar @Pepo48 based upon the above this isn't a windows issue. I think it's something we could avoid by switching the order of the checks in requiresReAugmentation. However we may also not want to encourage / allow application.properties files to appear in the provider jars. |
|
1.- What if the provider needs an application.properties file? And what about if it does not change? I suggest something like package definition, hash the class... Other solution, indeed. 2.- Just a warning and allow only the first/latest one, by deployment date? And a suggestion: Why not to include a kind-of "custom providers" interface, like "Client scopes"? "Custom Mappers" menu item, for global management... |
The issue is that quarkus uses a flat classloader and keycloak already includes an application.properties file at the root of the classpath. Only one of the application.properties files will eventually be used. We do not document and would not support trying to override the default configuration supplied by the keycloak application.properties file in this way. So if you need a specific configuration file for you provider jar, it should be named in such a way as to not conflict with other keycloak or provider classpath entries.
Please separate that into an enhancement request.
Please separate that into an enhancement request. |
|
Done! 😉 ( close this issue if you want! ) |
|
Thanks @shawkins for further investigation. Since you already did the hard work, I would mention it in the guide. Feel free to reassign it to yourself if you want. |
|
@Pepo48 ok, let's move this to be a documentation issue in the backlog for now. |
|
@vmuzikar @Pepo48 we have two separate issues here. This is an issue due to a resource conflict with an internal dependency, #26396 is due to the order in which we're preforming the checks, as mentioned above. If we first check if the provider jars have changed, before checking the configuration, then #26396 mostly goes away. On the internal resource conflict if my undestanding is correct we won't be making things better by simply reversing the checks - we don't really want the user to shadow anything that's built-in via the provider jars correct? In the case of a class we should see a split package warning. But there is currently no warning for something like application.properties. I don't know if we want to spend any effort towards this, or just stick to the documetnation plan. |
Ah, right. I'll reopen #26396. Thanks for pointing that out.
Yeah, from my perspective users should use the built-in versions. We don't have a separate classpath for extensions.
IMHO, documentation will do for now... |
closes: keycloak#24459 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
closes: keycloak#24459 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
closes: keycloak#24459 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
closes: keycloak#24459 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
closes: keycloak#24459 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
keycloak-github-bot
bot
added
priority/important
vmuzikar
added
priority/normal
and removed
priority/important
closes: #24459 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
…k#27214) closes: keycloak#24459 Signed-off-by: Steve Hawkins <shawkins@redhat.com>
Before reporting an issue
Area
core
Describe the bug
Hi.
I'm working with KeyCloak (both, with a Docker image and a locally downloaded v22.0.5) and a custom mapper for OIDC Tokens.
Basically, I've used the code in this URL:
https://betterjavacode.com/programming/custom-protocol-mapper-keycloak
I've generated a JAR and put it into the /opt/keycloak/providers folder. Let's call it: "
provider-cpm_1.0.0.jar".Later, I've used it (I can see it as a mapper) in a custom scope within a own Realm's client. It works.
( KeyCloak in a docker container from: keycloak/keycloak:latest )
But, if I want to "uninstall" the mapper from KeyCloak, I just delete the file, as par:
https://www.keycloak.org/server/configuration-provider#_installing_and_uninstalling_a_provider
The problem is that, if I stop the container and start it again, then a nice "
NoSuchFileException" error message comes:Any idea of how to avoid this situation?
Many thanks in advance.
Version
22.0.5
Expected behavior
The "
build" step, just before "start | start-dev" should take care of "non-existing" providers and do not stop server to initiate.Maybe, throwing an error message in the log file, but not stop it; or providing an alternate way to start or instructions on how to remediate the bug.
Actual behavior
Breaks the startup process, and the eserver is not starting
How to Reproduce?
/providers" folder.Anything else?
No response
The text was updated successfully, but these errors were encountered: