from version 1.13.0 Vault MicroProfile Config Source stopped working #16123

hernael · 2021-03-30T19:24:14Z

Describe the bug
from version 1.13.0 Vault MicroProfile Config Source stopped working to use vault with Databases (Vault MicroProfile Config Source) and OIDC

Expected behavior
quarkus fetched vault kv values first and then try verificted dependent properties. In version 1.12.X works fine, but in 1.13.0 does not work

Actual behavior
on startup throw error:

ERROR [io.qua.dep.dev.IsolatedDevModeMain] (main) Failed to start quarkus: java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step io.quarkus.datasource.deployment.devservices.DevServicesDatasourceProcessor#launchDatabases threw an exception: java.util.NoSuchElementException: SRCFG00011: Could not expand value database.url in property quarkus.datasource.reactive.url
	at io.smallrye.config.ExpressionConfigSourceInterceptor.lambda$getValue$0(ExpressionConfigSourceInterceptor.java:44)
	at io.smallrye.common.expression.ExpressionNode.emit(ExpressionNode.java:22)
	at io.smallrye.common.expression.Expression.evaluateException(Expression.java:56)
	at io.smallrye.common.expression.Expression.evaluate(Expression.java:70)
	at io.smallrye.config.ExpressionConfigSourceInterceptor.getValue(ExpressionConfigSourceInterceptor.java:37)
	at io.smallrye.config.ExpressionConfigSourceInterceptor.getValue(ExpressionConfigSourceInterceptor.java:18)
	at io.smallrye.config.SmallRyeConfigSourceInterceptorContext.proceed(SmallRyeConfigSourceInterceptorContext.java:20)
	at io.smallrye.config.SmallRyeConfig.getConfigValue(SmallRyeConfig.java:253)
	at io.smallrye.config.SmallRyeConfig.getValue(SmallRyeConfig.java:210)
	at io.smallrye.config.SmallRyeConfig.getOptionalValue(SmallRyeConfig.java:270)
	at io.quarkus.datasource.deployment.spi.DevServicesDatasourceConfigurationHandlerBuildItem$4.test(DevServicesDatasourceConfigurationHandlerBuildItem.java:101)
	at io.quarkus.datasource.deployment.spi.DevServicesDatasourceConfigurationHandlerBuildItem$4.test(DevServicesDatasourceConfigurationHandlerBuildItem.java:97)
	at io.quarkus.datasource.deployment.devservices.DevServicesDatasourceProcessor.startDevDb(DevServicesDatasourceProcessor.java:194)
	at io.quarkus.datasource.deployment.devservices.DevServicesDatasourceProcessor.launchDatabases(DevServicesDatasourceProcessor.java:103)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at io.quarkus.deployment.ExtensionLoader$2.execute(ExtensionLoader.java:920)
	at io.quarkus.builder.BuildContext.run(BuildContext.java:277)
	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2415)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1452)
	at java.base/java.lang.Thread.run(Thread.java:829)
	at org.jboss.threads.JBossThread.run(JBossThread.java:501)

	at io.quarkus.runner.bootstrap.AugmentActionImpl.runAugment(AugmentActionImpl.java:396)
	at io.quarkus.runner.bootstrap.AugmentActionImpl.createInitialRuntimeApplication(AugmentActionImpl.java:269)
	at io.quarkus.runner.bootstrap.AugmentActionImpl.createInitialRuntimeApplication(AugmentActionImpl.java:66)
	at io.quarkus.deployment.dev.IsolatedDevModeMain.firstStart(IsolatedDevModeMain.java:79)
	at io.quarkus.deployment.dev.IsolatedDevModeMain.accept(IsolatedDevModeMain.java:378)
	at io.quarkus.deployment.dev.IsolatedDevModeMain.accept(IsolatedDevModeMain.java:56)
	at io.quarkus.bootstrap.app.CuratedApplication.runInCl(CuratedApplication.java:127)
	at io.quarkus.bootstrap.app.CuratedApplication.runInAugmentClassLoader(CuratedApplication.java:84)
	at io.quarkus.deployment.dev.DevModeMain.start(DevModeMain.java:144)
	at io.quarkus.deployment.dev.DevModeMain.main(DevModeMain.java:63)
Caused by: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step io.quarkus.datasource.deployment.devservices.DevServicesDatasourceProcessor#launchDatabases threw an exception: java.util.NoSuchElementException: SRCFG00011: Could not expand value database.url in property quarkus.datasource.reactive.url
	at io.smallrye.config.ExpressionConfigSourceInterceptor.lambda$getValue$0(ExpressionConfigSourceInterceptor.java:44)
	at io.smallrye.common.expression.ExpressionNode.emit(ExpressionNode.java:22)
	at io.smallrye.common.expression.Expression.evaluateException(Expression.java:56)
	at io.smallrye.common.expression.Expression.evaluate(Expression.java:70)
	at io.smallrye.config.ExpressionConfigSourceInterceptor.getValue(ExpressionConfigSourceInterceptor.java:37)
	at io.smallrye.config.ExpressionConfigSourceInterceptor.getValue(ExpressionConfigSourceInterceptor.java:18)
	at io.smallrye.config.SmallRyeConfigSourceInterceptorContext.proceed(SmallRyeConfigSourceInterceptorContext.java:20)
	at io.smallrye.config.SmallRyeConfig.getConfigValue(SmallRyeConfig.java:253)
	at io.smallrye.config.SmallRyeConfig.getValue(SmallRyeConfig.java:210)
	at io.smallrye.config.SmallRyeConfig.getOptionalValue(SmallRyeConfig.java:270)
	at io.quarkus.datasource.deployment.spi.DevServicesDatasourceConfigurationHandlerBuildItem$4.test(DevServicesDatasourceConfigurationHandlerBuildItem.java:101)
	at io.quarkus.datasource.deployment.spi.DevServicesDatasourceConfigurationHandlerBuildItem$4.test(DevServicesDatasourceConfigurationHandlerBuildItem.java:97)
	at io.quarkus.datasource.deployment.devservices.DevServicesDatasourceProcessor.startDevDb(DevServicesDatasourceProcessor.java:194)
	at io.quarkus.datasource.deployment.devservices.DevServicesDatasourceProcessor.launchDatabases(DevServicesDatasourceProcessor.java:103)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at io.quarkus.deployment.ExtensionLoader$2.execute(ExtensionLoader.java:920)
	at io.quarkus.builder.BuildContext.run(BuildContext.java:277)
	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2415)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1452)
	at java.base/java.lang.Thread.run(Thread.java:829)
	at org.jboss.threads.JBossThread.run(JBossThread.java:501)

	at io.quarkus.builder.Execution.run(Execution.java:116)
	at io.quarkus.builder.BuildExecutionBuilder.execute(BuildExecutionBuilder.java:79)
	at io.quarkus.deployment.QuarkusAugmentor.run(QuarkusAugmentor.java:153)
	at io.quarkus.runner.bootstrap.AugmentActionImpl.runAugment(AugmentActionImpl.java:394)
	... 9 more
Caused by: java.util.NoSuchElementException: SRCFG00011: Could not expand value database.url in property quarkus.datasource.reactive.url
	at io.smallrye.config.ExpressionConfigSourceInterceptor.lambda$getValue$0(ExpressionConfigSourceInterceptor.java:44)
	at io.smallrye.common.expression.ExpressionNode.emit(ExpressionNode.java:22)
	at io.smallrye.common.expression.Expression.evaluateException(Expression.java:56)
	at io.smallrye.common.expression.Expression.evaluate(Expression.java:70)
	at io.smallrye.config.ExpressionConfigSourceInterceptor.getValue(ExpressionConfigSourceInterceptor.java:37)
	at io.smallrye.config.ExpressionConfigSourceInterceptor.getValue(ExpressionConfigSourceInterceptor.java:18)
	at io.smallrye.config.SmallRyeConfigSourceInterceptorContext.proceed(SmallRyeConfigSourceInterceptorContext.java:20)
	at io.smallrye.config.SmallRyeConfig.getConfigValue(SmallRyeConfig.java:253)
	at io.smallrye.config.SmallRyeConfig.getValue(SmallRyeConfig.java:210)
	at io.smallrye.config.SmallRyeConfig.getOptionalValue(SmallRyeConfig.java:270)
	at io.quarkus.datasource.deployment.spi.DevServicesDatasourceConfigurationHandlerBuildItem$4.test(DevServicesDatasourceConfigurationHandlerBuildItem.java:101)
	at io.quarkus.datasource.deployment.spi.DevServicesDatasourceConfigurationHandlerBuildItem$4.test(DevServicesDatasourceConfigurationHandlerBuildItem.java:97)
	at io.quarkus.datasource.deployment.devservices.DevServicesDatasourceProcessor.startDevDb(DevServicesDatasourceProcessor.java:194)
	at io.quarkus.datasource.deployment.devservices.DevServicesDatasourceProcessor.launchDatabases(DevServicesDatasourceProcessor.java:103)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at io.quarkus.deployment.ExtensionLoader$2.execute(ExtensionLoader.java:920)
	at io.quarkus.builder.BuildContext.run(BuildContext.java:277)
	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2415)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1452)
	at java.base/java.lang.Thread.run(Thread.java:829)
	at org.jboss.threads.JBossThread.run(JBossThread.java:501)

Hi @radcortez, @vsevel: please note that this error is equal to old #14707 but instead the releases 1.11.0 it happens in new releases 1.13.0

The text was updated successfully, but these errors were encountered:

quarkus-bot · 2021-03-30T19:24:17Z

/cc @vsevel

radcortez · 2021-03-30T20:05:38Z

Does this only happens with the Vault source? Can you try without it?

hernael · 2021-03-30T20:41:36Z

without vault?
sorry but i don't understand well the test that you say me

radcortez · 2021-03-30T20:49:21Z

Yes, are you able to remove Vault and set up the required configuration in application.properties? I'm trying to rule out if this is something Vault specific or it it is general.

I supposed that database.url is set up in Vault. You may try to set something up in application.properties. It doesn't need to be valid, we just need to make sure that the expansion works.

hernael · 2021-03-30T21:04:51Z

ok, i only replace the db properties referenced by Vault in application.properties (quarkus.datasource.reactive.url, quarkus.datasource.password, quarkus.datasource.username) by equivalent conection params and this works fine.

Additional info: vault oidc properties in application.properties work fine.

vsevel · 2021-03-30T21:39:52Z

In 1.13.CR1 I only added #15617 and #15353
does not seem to be related

radcortez · 2021-03-30T21:50:54Z

Do you have other properties set up in Vault? Is this only failing for expanded properties for Vault? Can you provide me a print of ConfigProvider.getConfig().getConfigSources()? I'm interested in the types, names and order of the sources.

hernael · 2021-03-30T22:10:34Z

the properties set for vault are:

quarkus.datasource.username
quarkus.datasource.password
quarkus.datasource.reactive.url
quarkus.oidc.auth-server-url
quarkus.oidc.client-id

Q: Is this only failing for expanded properties for Vault?
R: looks like only the quarkus.datasource expanded for Vault fails

ConfigProvider.getConfig().getConfigSources():

...

hernael · 2021-03-30T22:51:53Z

please best use this data:

...

I'm not sure but data in previous post maybe is for 1.10.5. Sorry...

radcortez · 2021-03-30T23:18:47Z

@hernael I took a copy and edited the prints, since these contain some environment variables and I don't want you to accidentally expose any sensitive information. I'll have a look tomorrow.

radcortez · 2021-03-31T16:54:23Z

Hi @hernael, I didn't find anything suspicious in the data. Could you iterate over ConfigProvider.getConfig().getConfigSources(); and when you get to the VaultConfigSource try to retrieve the keys directly from the source?

I've tried locally with a Vault docker image and a few properties, including starting up a database with expanded properties coming from Vault and it worked fine. Maybe something else in your setup is breaking the lookup.

hernael · 2021-03-31T21:24:42Z

I just updated a demo project that i have hosted for issues purposes:

https://github.com/hernael/reactive-demo

Maybe seeing this helps us a little better. Please note that i started the app with intellij in dev mode (just click run in idea)

radcortez · 2021-04-01T12:10:00Z

Ok, let me check

radcortez · 2021-04-01T12:44:03Z

Ok, I believe this is related with #14960 and more specifically with:

quarkus/extensions/datasource/deployment-spi/src/main/java/io/quarkus/datasource/deployment/spi/DevServicesDatasourceConfigurationHandlerBuildItem.java

Lines 98 to 108 in f4cba6a

    
           @Override 
        
           public boolean test(String dsName) { 
        
               if (dsName == null) { 
        
                   return ConfigProvider.getConfig().getOptionalValue("quarkus.datasource.reactive.url", String.class) 
        
                           .isPresent(); 
        
               } else { 
        
                   return ConfigProvider.getConfig() 
        
                           .getOptionalValue("quarkus.datasource.\"" + dsName + "\".reactive.url", String.class) 
        
                           .isPresent(); 
        
               } 
        
           }

This checks if a DS configuration is available to start / not start a database if you need one. But the check happens in build time when the VaultConfigSource is not available so the expansion fails. For code purpose, it is irrelevant if the value is expanded, since we are only interested if the value is set. I'll provide a fix for this.

In the meanwhile, I recommend to workaround this by setting an empty default in the expression: quarkus.datasource.reactive.url=${database.url:} (note the final colon in the expansion name).

Sorry for the inconvenience.

hernael · 2021-04-01T14:50:48Z

ok @radcortez, thanks for the solution!

gsmet · 2021-04-01T15:40:45Z

@radcortez any chance we could have a fix by Tuesday evening?

radcortez · 2021-04-01T15:43:08Z

@radcortez any chance we could have a fix by Tuesday evening?

Yes, doing it right now.

hernael added the kind/bug Something isn't working label Mar 30, 2021

quarkus-bot bot added the area/vault label Mar 30, 2021

gsmet added this to the 1.13.1.Final milestone Apr 1, 2021

radcortez mentioned this issue Apr 1, 2021

API to check if property exists #16189

Merged

gsmet closed this as completed in #16189 Apr 6, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

from version 1.13.0 Vault MicroProfile Config Source stopped working #16123

from version 1.13.0 Vault MicroProfile Config Source stopped working #16123

hernael commented Mar 30, 2021

quarkus-bot bot commented Mar 30, 2021

radcortez commented Mar 30, 2021

hernael commented Mar 30, 2021 •

edited

radcortez commented Mar 30, 2021

hernael commented Mar 30, 2021

vsevel commented Mar 30, 2021

radcortez commented Mar 30, 2021

hernael commented Mar 30, 2021 •

edited by radcortez

hernael commented Mar 30, 2021 •

edited by radcortez

radcortez commented Mar 30, 2021 •

edited

radcortez commented Mar 31, 2021

hernael commented Mar 31, 2021

radcortez commented Apr 1, 2021

radcortez commented Apr 1, 2021

hernael commented Apr 1, 2021

gsmet commented Apr 1, 2021

radcortez commented Apr 1, 2021

from version 1.13.0 Vault MicroProfile Config Source stopped working #16123

from version 1.13.0 Vault MicroProfile Config Source stopped working #16123

Comments

hernael commented Mar 30, 2021

quarkus-bot bot commented Mar 30, 2021

radcortez commented Mar 30, 2021

hernael commented Mar 30, 2021 • edited

radcortez commented Mar 30, 2021

hernael commented Mar 30, 2021

vsevel commented Mar 30, 2021

radcortez commented Mar 30, 2021

hernael commented Mar 30, 2021 • edited by radcortez

hernael commented Mar 30, 2021 • edited by radcortez

radcortez commented Mar 30, 2021 • edited

radcortez commented Mar 31, 2021

hernael commented Mar 31, 2021

radcortez commented Apr 1, 2021

radcortez commented Apr 1, 2021

hernael commented Apr 1, 2021

gsmet commented Apr 1, 2021

radcortez commented Apr 1, 2021

hernael commented Mar 30, 2021 •

edited

hernael commented Mar 30, 2021 •

edited by radcortez

hernael commented Mar 30, 2021 •

edited by radcortez

radcortez commented Mar 30, 2021 •

edited