Breaking change in 3.1.8: using config.import with gives "[//] give is not a valid HTTP URL"

[MarkvanOsch]

Describe the bug

After upgrading from 3.1.7 to 3.1.8 using config.import with optional and using default "://" gives a "[//] is not a valid HTTP URL error".

Stacktrace:

Caused by: java.lang.IllegalArgumentException: [//] is not a valid HTTP URL
	at org.springframework.web.util.UriComponentsBuilder.fromHttpUrl(UriComponentsBuilder.java:316)

Sample
Using this config in application.yml:

spring:
    config.import:
      - optional:vault://
      - optional:configserver:${SPRING_CLOUD_CONFIG_URI://}

As reference this should still be valid use:
Conditionally enable/disable Vault Configuration
In some cases, it can be required to launch an application without Vault. You can express whether a Vault config location should be optional or mandatory (default) through the location string:

optional:vault:// (default location)

See: https://docs.spring.io/spring-cloud-vault/docs/current/reference/html/config-data.html

Using these dependencies:

<spring-boot.version>2.7.13</spring-boot.version>
<spring-cloud.version>2021.0.8</spring-cloud.version>

[MarkvanOsch]

@ryanjbaxter Hi Ryan, this codechange 9f3ad4f is breaking the use of for example "optional:vault://" in the spring.config.import property. The "//" as uri should be allowed also.

The urisHashCode calls UriComponentsBuilder.fromHttpUrl which results in above error.

[MarkvanOsch]

@ryanjbaxter will this be also added to spring-cloud-dependencies 2021.0.9?

[ryanjbaxter]

Correct the fix went into the 3.1.x, 4.0.x, and main branches

[MarkvanOsch]

@ryanjbaxter the issue still occurs when using the newest spring-cloud-config-client 4.0.4, but now as a WARN:

WARN 60490 --- [main] o.s.c.c.c.ConfigServerConfigDataLoader : Could not locate PropertySource ([ConfigServerConfigDataResource@7c3ebc6b uris = array<String>['//'], optional = true, profiles = 'default']): Invalid URL: //

Using these dependencies:

<spring-boot.version>3.1.2</spring-boot.version>
<spring-cloud.version>2022.0.4</spring-cloud.version>

Using spring-cloud-config-client 4.0.2 (spring cloud 2022.0.2) this WARN is not thrown.

[ryanjbaxter]

@MarkvanOsch the problem is actually this line optional:configserver:${SPRING_CLOUD_CONFIG_URI://}. // is not valid for the config server (like it is vault). I think what you want is optional:configserver:${SPRING_CLOUD_CONFIG_URI:}

[MarkvanOsch]

@ryanjbaxter I tried with several setups, but above fails with:

java.lang.IllegalStateException: Unable to load config data from 'optional:configserver:'

Caused by: java.lang.IllegalStateException: File extension is not known to any PropertySourceLoader. If the location is meant to reference a directory, it must end in '/' or File.separator

It seems like the behaviour of import checking is changed. This is my setup that I use and works fine in cloud env, only in local dev and test gives a WARN.

This works fine in local dev with:
Spring boot 3.1.2
Spring cloud 2022.0.2
Spring cloud config client 4.0.2

But gives a WARN with:
Spring boot 3.1.2
Spring cloud 2022.0.3 or 2022.0.4
Spring cloud config client 4.0.3 or 4.0.4

Is this a different issue maybe?

[ryanjbaxter]

If you remove ‘//‘ from the spring.config.import statement does the warning go away with 2202.0.4?

[MarkvanOsch]

No, then I get this ERROR:

java.lang.IllegalStateException: Unable to load config data from 'optional:configserver:'

Caused by: java.lang.IllegalStateException: File extension is not known to any PropertySourceLoader. If the location is meant to reference a directory, it must end in '/' or File.separator

[ryanjbaxter]

And you have spring cloud config on the class path and it’s enabled? Can you provide a sample that reproduces the problem?

[MarkvanOsch]

I think this discussion is now active again: #1877

So I should change my application.yml to this setup and only do the import on my cloud profiles. It that the way forward?

Or even this because the environment variable should be available in the cloud environments:

Still curious why the WARN is now happening with when using spring.cloud.import in the default profile:
Spring cloud config client 4.0.3 or 4.0.4

[ryanjbaxter]

The last image seems right.

Hard for me to say why you see the warning now with the default profile, I need an example to look at. // was never a valid was to tell the conflig client to use the default url like it is for vault

[MarkvanOsch]

Thanks, I will provide a sample to reproduce.

[MarkvanOsch]

Sample project to reproduce, with readme.

spring_cloud_config_reproduce_2304.zip

[ryanjbaxter]

Thanks!

This is kind of interesting but the behavior is now expected.

Spring Boot's config data loading happens in 2 phases:

1. Load configuration without any profiles active in order to collect any activated profiles
2. Load configuration with all the activated profiles

Prior to 2022.0.3 the config client skipped 1 and only did 2. Now since we also do 1 your import statement is being run by the config client to try and load configuration without the dev profile active and you see the warning. Then after the dev profile gets active the config client config data loader doesn't run in phase 2.

I think the reason why you are seeing Unable to load config data from 'optional:configserver:' when you remove the // is because when the // is present it is matching one of the Spring Boot config data loaders and that is handling the import statement.

These are actually 2 completely unrelated behaviors :)

[MarkvanOsch]

Thanks Ryan, it's clear to me now. 👍

In our setup I will go forward to moving the config.import to the specific cloud profile. So not use it in the default.

[ENate]

I am trying to load vault using spring.config.import; vault:// in the spring config server's application.properties file and still get

Unable to load data from vault://

Caused by: java.lang.IllegalStateException: File extension is not known to any PropertySourceLoader. If the location is meant to reference a directory, it must end in '/' or File.separator

Can anyone help me get why this is still happening on local dev environment? I am using spring cloud 2023.0.0 and boot 3.2.+ ? Thanks

[MarkvanOsch]

@ENate Hi, this is the setup we now use which solves the earlier issues. It supports local development (without vault/configserver) and cloud deployments (with vault/configserver) using profiles. Maybe you can use it to double check your setup?

[ENate]

Hi @MarkvanOsch thanks. I tried to import both vault and config server properties for other services but still getting an error with the property source not found. The issue is: I locked my config server using spring security (and implemented login calls from a database). Next, I will also want services to use the login details which I provided via vault to load their properties from config server. I do not know whether it is best to fetch login details directly from the config server database. But I still feel it is sufficient to provide the login for other services via vault for them to be picked up. But the issue could be with the property source not being loaded. The config--server runs fine. The main issue is the inability for other services to pick up the secrets from vault and login to the config server - since it seems they all fail to load vault. Thanks again