Integrating with Spring Cloud Config Server #501

thiagolocatelli · 2018-08-17T14:28:07Z

I want to migrate some of the applications I have written with Spring Boot but I would like to keep using the config servers I have. Looking at the documentation, I found this: "Alternatively one can register a PropertySourceLoader by creating a META-INF/services/io.micronaut.context.env.PropertySourceLoader containing a reference to the class name of the PropertySourceLoader".

I am creating my own spring config server client for Micronaut to read the properties from the config server, should I just go with my own implementation of PropertySourceLoader or should I follow the same approach used for Eureka and Consul in the discovery-client module?

graemerocher · 2018-08-17T14:51:58Z

The latter. You should implement the ConfigurationClient interface

thiagolocatelli · 2018-08-17T14:53:06Z

I will look into that. Thank you @graemerocher

thiagolocatelli · 2018-08-21T02:47:56Z

Started working on this: https://github.com/thiagolocatelli/micronaut-spring-client

Still can't figure out why the method getPropertySources from SpringCloudConfigurationClient is not being invoked. I may be missing something. Decided to create the code in another repo and once it's stable and working 100% I will merge it into the oficial repo.

thiagolocatelli · 2018-08-21T03:16:16Z

with the help form @jameskleeh I identified the discovery-client library was missing. After adding it to the project, my SpringCloudConfigurationClient started being invoked.

thiagolocatelli · 2018-08-21T13:54:53Z

So the client for Spring Cloud config is working pretty well. I have just one question before I start writing the unit tests. The PropertySource.of method takes a priority. Which priority is loaded first: lowest or highest?

Let's say I have a property "application.location" with priorities 10 and 20, which one will have the final value in the application context?

jameskleeh · 2018-08-21T14:02:37Z

@thiagolocatelli I believe the higher order will override config values with a lower order

thiagolocatelli · 2018-08-21T14:04:10Z

@jameskleeh sounds good James. That's how I implemented. Thank you.

thiagolocatelli · 2018-08-23T17:54:29Z

Progress: discovery client for spring cloud config server is working properly

13:49:12.321 [main] INFO  i.m.context.env.DefaultEnvironment - Established active environments: [dev, docker]
13:49:13.493 [main] INFO  i.m.d.s.c.SpringCloudConfigConfigurationClient - Config Server endpoint: http://127.0.0.1:8888/configserver
13:49:13.493 [main] INFO  i.m.d.s.c.SpringCloudConfigConfigurationClient - Application name: spring-config-sample, application profiles: dev,docker
13:49:14.211 [nioEventLoopGroup-1-2] DEBUG i.m.http.client.DefaultHttpClient - Sending HTTP Request: GET /configserver/spring-config-sample/dev%2Cdocker
13:49:14.211 [nioEventLoopGroup-1-2] DEBUG i.m.http.client.DefaultHttpClient - Chosen Server: 127.0.0.1(8888)
13:49:15.423 [nioEventLoopGroup-1-2] INFO  i.m.d.s.c.SpringCloudConfigConfigurationClient - Property source entry: http://*****/application-docker.properties
13:49:15.423 [nioEventLoopGroup-1-2] INFO  i.m.d.s.c.SpringCloudConfigConfigurationClient - Property source entry: http://*****/application-dev.properties
13:49:15.428 [nioEventLoopGroup-1-2] INFO  i.m.d.s.c.SpringCloudConfigConfigurationClient - Property source entry: http://*****/spring-config-sample.properties
13:49:15.433 [nioEventLoopGroup-1-2] INFO  i.m.d.s.c.SpringCloudConfigConfigurationClient - Property source entry: http://*****/application.properties
13:49:15.465 [main] INFO  i.m.d.c.c.DistributedPropertySourceLocator - Resolved 3 configuration sources from client: compositeConfigurationClient(spring-cloud-config-client)
13:49:15.621 [main] INFO  io.micronaut.runtime.Micronaut - Startup completed in 3289ms. Server Running: http://localhost:8080

jameskleeh · 2018-08-23T19:03:02Z

@thiagolocatelli How are you ordering the property sources when both application and app specific config is specified for the environment?

We should probably standardize and document what order the property sources should return in

I think how it currently works for consul is:

application
foo
application,dev
foo,dev

thiagolocatelli · 2018-08-23T19:05:59Z

I am currently ordering based on the response sent by the config server, which seems to the correct order to be used. The first one gets the highest priority and the subsequent ones get lowers priority.

jameskleeh · 2018-08-23T19:06:40Z

@thiagolocatelli In your logs there is no example with the application name and environment

thiagolocatelli · 2018-08-23T19:09:01Z

there is, in my example I am using -Dmicronaut.environments=dev,docker

13:49:13.493 [main] INFO  i.m.d.s.c.SpringCloudConfigConfigurationClient - Application name: spring-config-sample, application profiles: dev,docker
13:49:15.423 [nioEventLoopGroup-1-2] INFO  i.m.d.s.c.SpringCloudConfigConfigurationClient - Property source entry: http://*****/application-docker.properties
13:49:15.423 [nioEventLoopGroup-1-2] INFO  i.m.d.s.c.SpringCloudConfigConfigurationClient - Property source entry: http://*****/application-dev.properties
13:49:15.428 [nioEventLoopGroup-1-2] INFO  i.m.d.s.c.SpringCloudConfigConfigurationClient - Property source entry: http://*****/spring-config-sample.properties

jameskleeh · 2018-08-23T19:10:34Z

@thiagolocatelli There isn't a property source for spring-config-sample specific to an environment

thiagolocatelli · 2018-08-23T19:12:22Z

if I had included a file spring-config-sample-{env}.properties into my config server it would have been picked up by the client. My config server only contains application.properties, application-{env}.properties and spring-config-sample.properties.

jameskleeh · 2018-08-23T19:22:53Z

@thiagolocatelli Great! Have you verified that if that file were there it would be returned in the correct order?

thiagolocatelli · 2018-08-23T19:25:14Z

@jameskleeh working on that right now! will include the spring-config-sample-{env}.properties files there and see what comes up.

thiagolocatelli · 2018-08-23T19:59:52Z

@jameskleeh so I have included the following files to my config server:

spring-sample-config.properties
spring-sample-config-dev.properties
spring-sample-config-docker.properties

the response back from the config server was:

15:47:05.218 [main] INFO  i.m.d.s.c.SpringCloudConfigConfigurationClient - Application name: spring-config-sample, application profiles: dev,docker
15:47:07.782 [nioEventLoopGroup-1-2] DEBUG i.m.http.client.DefaultHttpClient - Sending HTTP Request: GET /configserver/spring-config-sample/dev%2Cdocker
15:47:11.051 [nioEventLoopGroup-1-2] INFO  i.m.d.s.c.SpringCloudConfigConfigurationClient - Property source entry: http://*****/spring-config-sample-docker.properties
15:47:11.051 [nioEventLoopGroup-1-2] INFO  i.m.d.s.c.SpringCloudConfigConfigurationClient - Property source entry: http://*****/application-docker.properties
15:47:11.052 [nioEventLoopGroup-1-2] INFO  i.m.d.s.c.SpringCloudConfigConfigurationClient - Property source entry: http://*****/spring-config-sample-dev.properties
15:47:11.052 [nioEventLoopGroup-1-2] INFO  i.m.d.s.c.SpringCloudConfigConfigurationClient - Property source entry: http://*****/application-dev.properties
15:47:11.052 [nioEventLoopGroup-1-2] INFO  i.m.d.s.c.SpringCloudConfigConfigurationClient - Property source entry: http://*****/spring-config-sample.properties
15:47:11.052 [nioEventLoopGroup-1-2] INFO  i.m.d.s.c.SpringCloudConfigConfigurationClient - Property source entry: http://*****/application.properties

so it seems to me that spring property source ordering is as follow, considering app name is 'foo' and environments are 'dev' and 'docker':

foo-docker.properties
application-docker.properties
foo-dev.properties
application-dev.properties
foo.properties
application.properties

I am taking the environments names from environment.getActiveNames(), which returns a java.util.Set, so no matter the order the developer wants, it can be changed due to the underlying implementation of Set, which is sorted by default (maybe this could be an enhancement and let the order provided in the micronaut.environment property be honored by the Micronaut).

jameskleeh · 2018-08-23T20:03:07Z

@thiagolocatelli Seems like it is returning them in the correct order. I think it would be wise to ensure that the order is correct locally to be future proof of changes in spring cloud.

It makes sense to me that the environment order should be a list and the order should be respected

thiagolocatelli · 2018-08-23T20:04:57Z

@jameskleeh I will open a separate issue on that and will do my best to send a PR with a fix. ok?

graemerocher · 2018-08-28T11:22:30Z

@thiagolocatelli PRs welcome

thiagolocatelli · 2018-08-28T14:01:48Z

@graemerocher PR will be sent soon. First I need to setup micronaut-core on my local machine and move the client code to micronaut-core, right now its a library inside my project, its not inside micronaut yet. Second, I need to write unit tests for it and then I will be ready for the PR.

Second phase would be include Vault as part of the process too, but I guess spring cloud config is a good beginning. Sorry for the delay.

graemerocher · 2018-08-28T14:23:02Z

No worries, thanks for contributing!

thiagolocatelli · 2018-08-30T11:37:59Z

@graemerocher I have made the client available at https://github.com/thiagolocatelli/micronaut-spring-client

I am still trying to figure out how I can use the SPRING_CLOUD_CONFIG_URI environment variable. The default scenario should be: first use the environment variable, if not available, then use spring.cloud.config.uri property, if that is also not available, default to http://localhost:8888 (default config server uri when deployed locally). Once I figure how to achieve this, I will merge that code into micronaut-core and send the PR.

If you have any suggestion on how to achieve that, let me know.

jameskleeh · 2018-08-30T13:49:47Z

@thiagolocatelli A PR was just merged yesterday to allow backticks.

${SPRING_CLOUD_CONFIG_URI:spring.cloud.config.uri:`http://localhost:8888`}

thiagolocatelli · 2018-08-30T13:59:09Z

@jameskleeh per our discussion over Gitter, we found out that Micronaut wasnt evaluating the second element in the expression. It was returning a constant instead of trying to extract the value from the application context. ${SPRING_CLOUD_CONFIG_URI:spring.cloud.config.uri} returns 'spring.cloud.config.uri' instead of the value for it. I had opened an issue #516 but it was closed a couple of days ago. Even though I have spring.cloud.config.uri in the bootstrap.yml, micronaut is not evaluating it.

In your example ${SPRING_CLOUD_CONFIG_URI:spring.cloud.config.uri:`http://localhost:8888`}, it will never return localhost:8888 because the constant 'spring.cloud.config.uri' will be returned if env variable is not there.

jameskleeh · 2018-08-30T14:00:35Z

@thiagolocatelli That isn't true, only the last value is treated as a constant

thiagolocatelli · 2018-08-30T14:11:51Z

@jameskleeh ok, Thanks. I will wait for the next release with the PR sent to allow backticks to keep working on the config client.

datamaskin · 2018-09-26T14:56:35Z

We have a SCCS running in house using ${GIT_URL} defined in the shell which SCCS seems to recognize from application.yml or a default. Would really like to see a MN profile for creating SCCS clients. Thanks.

MichelSchudel · 2018-11-09T18:17:15Z

@graemerocher I'd like to take a crack at this and help you with this one, since we would like to have this feature so we can embed new MN services in our Spring Cloud ecosystem. The ability to use SCC for MN configuration would be very helpful to us. So, what's still open exactly, the ability to use SCC (a Client, I saw @thiagolocatelli did a lot of work already), or is creating a profile the only work left?

graemerocher · 2018-11-09T19:06:04Z

We have made no progress on this issue. I don’t know what the state of @thiagolocatalli’s work is. Any help would be appreciated!

thiagolocatelli · 2018-11-10T16:08:20Z

@MichelSchudel Hey Michel, I have completed the code, its available in this repo: https://github.com/thiagolocatelli/micronaut-spring-client

I just havent had the chance to merge my changes back into the official repo, you can give it a try. It has been working for us.

MichelSchudel · 2018-11-10T16:11:30Z

Sure, I will take a look and give it a try.

…

On Sat, Nov 10, 2018, 17:08 Thiago Locatelli ***@***.*** wrote: @MichelSchudel <https://github.com/MichelSchudel> Hey Michel, I have completed the code, its available in this repo: https://github.com/thiagolocatelli/micronaut-spring-client I just havent had the chance to merge my changes back into the official repo, you can give it a try. It has been working for us. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#501 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFxMefWA4EN3R7ugzm0BoAnj1jBN_ZQbks5utvn3gaJpZM4WBqZi> .

mdkhwajams · 2018-11-16T01:57:19Z

Great work! Thanks for this integration with spring cloud config server looking forward to utilize it.

MichelSchudel · 2018-11-16T10:00:26Z

Working on it, having some issues getting all unit tests to run from micronaut-core master branch. Some AWS / EC2 tests in discovery client fail.

…

On Fri, Nov 16, 2018 at 2:57 AM mdkhwajams ***@***.***> wrote: Great work! Thanks for this integration with spring cloud config server looking forward to utilize it. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#501 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFxMeYN3EQ0kQ_wC_quSomqsQaW6Ni_5ks5uvhuDgaJpZM4WBqZi> .

-- -------------------------------- Michel Schudel email: michel.schudel@gmail.com website: www.michelschudel.nl twitter: http://twitter.com/MichelSchudel facebook: http://www.facebook.com/michel.schudel -------------------------------

thiagolocatelli · 2018-11-28T23:35:26Z

@MichelSchudel Did you get any lucky integrating with the micronaut-core project?

MichelSchudel · 2018-11-28T23:38:36Z

Working on getting all existing micronaut-core unit tests to run on windows 10... but your solution is fine. I just added some integration tests to test your solution. Still have to integrate it with micronaut-core through pull request, though.

…

On Thu, Nov 29, 2018, 00:35 Thiago Locatelli ***@***.*** wrote: @MichelSchudel <https://github.com/MichelSchudel> Did you get any lucky integrating with the micronaut-core project? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#501 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFxMedAW01LJLjT7U45kqHyzkf6rNS0Mks5uzx3DgaJpZM4WBqZi> .

thiagolocatelli · 2018-11-29T00:15:33Z

Awesome! Thank you

graemerocher · 2018-12-20T07:31:00Z

@MichelSchudel maybe you want to send the PR anyway and we can clean it up if necessary

MichelSchudel · 2018-12-20T09:46:50Z

@graemerocher is there any way I can send you a private message? Twitter would be preferrable (follow me)

thiagolocatelli · 2018-12-26T13:38:35Z

@MichelSchudel were you able to send the PR? If not, I will work on sending the PR by Friday.

MichelSchudel · 2018-12-26T14:08:39Z

Hi @thiagolocatelli,
I have updated your client project (through a fork) to work with the recent version of micronaut master and added an integration test. I sent you a pull request, plz review and merge if you like.

Due to a tragic familiy event I have been unable to cast your code into the micronaut-core java and test package structures. So, you still have to do that. Sorry but I have no time for that right now. So if you do, please use my changes, and make a pull request, by all means.

thiagolocatelli · 2018-12-26T14:17:19Z

@MichelSchudel sorry to hear about the family event. I will take care now and send @graemerocher a PR into their repo. I will be adding the code to their discovery module.

Thank you for the changes you made.

MichelSchudel · 2018-12-26T14:22:15Z

You're welcome, thank you for the apprecation. The integration test is a Java test, I don't know if it should be a groovy test, since most micronaut core tests are. See what you can use, glad to have helped.

…

On Wed, Dec 26, 2018, 15:17 Thiago Locatelli ***@***.*** wrote: @MichelSchudel <https://github.com/MichelSchudel> sorry to hear about the family event. I will take care now and send @graemerocher <https://github.com/graemerocher> a PR into their repo. I will be adding the code to their discovery module. Thank you for the changes you made. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#501 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFxMeRUwNVwm5VEhkg6zD0OkjpPKOfF8ks5u84TygaJpZM4WBqZi> .

thiagolocatelli · 2018-12-26T16:39:39Z

@graemerocher PR sent

MichelSchudel · 2019-01-03T15:11:41Z

👌

…

On Thu, Jan 3, 2019, 15:45 Graeme Rocher ***@***.*** wrote: Closed #501 <#501>. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#501 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFxMeWWervfet27JmqtDdcFwOXCZiQgAks5u_heNgaJpZM4WBqZi> .

jameskleeh assigned thiagolocatelli Aug 20, 2018

thiagolocatelli mentioned this issue Aug 23, 2018

DefaultEnvironment should honor the order of environment names #523

Closed

graemerocher added the help wanted Extra attention is needed label Oct 9, 2018

thiagolocatelli mentioned this issue Dec 26, 2018

Spring Cloud config server client #1065

Merged

graemerocher added the status: pr submitted A pull request has been submitted for the issue label Jan 2, 2019

graemerocher added a commit that referenced this issue Jan 3, 2019

Add documentation, tests and cleanup for Spring Cloud Config. Issue #501

bfcf63e

graemerocher added this to the 1.1.0 milestone Jan 3, 2019

graemerocher closed this as completed Jan 3, 2019

Integrating with Spring Cloud Config Server #501

Integrating with Spring Cloud Config Server #501

Comments

thiagolocatelli commented Aug 17, 2018 • edited

graemerocher commented Aug 17, 2018

thiagolocatelli commented Aug 17, 2018

thiagolocatelli commented Aug 21, 2018 • edited

thiagolocatelli commented Aug 21, 2018

thiagolocatelli commented Aug 21, 2018 • edited

jameskleeh commented Aug 21, 2018

thiagolocatelli commented Aug 21, 2018

thiagolocatelli commented Aug 23, 2018

jameskleeh commented Aug 23, 2018 • edited

thiagolocatelli commented Aug 23, 2018

jameskleeh commented Aug 23, 2018

thiagolocatelli commented Aug 23, 2018

jameskleeh commented Aug 23, 2018

thiagolocatelli commented Aug 23, 2018

jameskleeh commented Aug 23, 2018

thiagolocatelli commented Aug 23, 2018

thiagolocatelli commented Aug 23, 2018 • edited

jameskleeh commented Aug 23, 2018

thiagolocatelli commented Aug 23, 2018

graemerocher commented Aug 28, 2018

thiagolocatelli commented Aug 28, 2018

graemerocher commented Aug 28, 2018

thiagolocatelli commented Aug 30, 2018

jameskleeh commented Aug 30, 2018

thiagolocatelli commented Aug 30, 2018

jameskleeh commented Aug 30, 2018

thiagolocatelli commented Aug 30, 2018

datamaskin commented Sep 26, 2018

MichelSchudel commented Nov 9, 2018

graemerocher commented Nov 9, 2018

thiagolocatelli commented Nov 10, 2018

MichelSchudel commented Nov 10, 2018 via email

mdkhwajams commented Nov 16, 2018

MichelSchudel commented Nov 16, 2018 via email

thiagolocatelli commented Nov 28, 2018

MichelSchudel commented Nov 28, 2018 via email

thiagolocatelli commented Nov 29, 2018

graemerocher commented Dec 20, 2018

MichelSchudel commented Dec 20, 2018

thiagolocatelli commented Dec 26, 2018

MichelSchudel commented Dec 26, 2018

thiagolocatelli commented Dec 26, 2018

MichelSchudel commented Dec 26, 2018 via email

thiagolocatelli commented Dec 26, 2018

MichelSchudel commented Jan 3, 2019 via email

thiagolocatelli commented Aug 17, 2018 •

edited

thiagolocatelli commented Aug 21, 2018 •

edited

thiagolocatelli commented Aug 21, 2018 •

edited

jameskleeh commented Aug 23, 2018 •

edited

thiagolocatelli commented Aug 23, 2018 •

edited