Permalink
Browse files

Push events for config changes

Works from local repos or with explicit POST to /monitor with
path=<serviceId>, also supports webhooks from github and gitlab.
  • Loading branch information...
dsyer committed Sep 8, 2015
1 parent dc5097b commit 8d8e07a78e73c3e1e08f4277a76c6e1a4104f103
Showing with 1,339 additions and 9 deletions.
  1. +36 −1 docs/src/main/asciidoc/spring-cloud-config.adoc
  2. +8 −1 pom.xml
  3. +49 −0 spring-cloud-config-monitor/pom.xml
  4. +84 −0 ...ain/java/org/springframework/cloud/config/monitor/CompositePropertyPathNotificationExtractor.java
  5. +43 −0 ...r/src/main/java/org/springframework/cloud/config/monitor/EnvironmentMonitorAutoConfiguration.java
  6. +295 −0 ...nfig-monitor/src/main/java/org/springframework/cloud/config/monitor/FileMonitorConfiguration.java
  7. +66 −0 ...c/main/java/org/springframework/cloud/config/monitor/GithubPropertyPathNotificationExtractor.java
  8. +47 −0 ...c/main/java/org/springframework/cloud/config/monitor/GitlabPropertyPathNotificationExtractor.java
  9. +128 −0 ...d-config-monitor/src/main/java/org/springframework/cloud/config/monitor/PropertyPathEndpoint.java
  10. +38 −0 ...nfig-monitor/src/main/java/org/springframework/cloud/config/monitor/PropertyPathNotification.java
  11. +36 −0 ...tor/src/main/java/org/springframework/cloud/config/monitor/PropertyPathNotificationExtractor.java
  12. +3 −0 spring-cloud-config-monitor/src/main/resources/META-INF/spring.factories
  13. +79 −0 ...ava/org/springframework/cloud/config/monitor/CompositePropertyPathNotificationExtractorTests.java
  14. +77 −0 ...t/java/org/springframework/cloud/config/monitor/GithubPropertyPathNotificationExtractorTests.java
  15. +67 −0 ...t/java/org/springframework/cloud/config/monitor/GitlabPropertyPathNotificationExtractorTests.java
  16. +72 −0 ...fig-monitor/src/test/java/org/springframework/cloud/config/monitor/PropertyPathEndpointTests.java
  17. +161 −0 spring-cloud-config-monitor/src/test/resources/github.json
  18. +42 −0 spring-cloud-config-monitor/src/test/resources/gitlab.json
  19. +8 −7 ...ud-config-server/src/main/java/org/springframework/cloud/config/server/EnvironmentController.java
@@ -489,7 +489,7 @@ TIP: the `{name:value}` prefixes can also be added to plaintext posted
to the `/encrypt` endpoint, if you want to let the Config Server
handle all encryption as well as decryption.
=== Embedding the Config Server
== Embedding the Config Server
The Config Server runs best as a standalone application, but if you
need to you can embed it in another application. Just use the
@@ -505,6 +505,41 @@ own remote repository. The flag is off by default because it can delay
startup, but when embedded in another application it makes sense to
initialize the same way as any other application.
== Push Notifications and Spring Cloud Bus
Many source code repository providers (like Github or Gitlab for
instance) will notify you of changes in a repository through a
webhook. You can configure the webhook via the provider's user
interface as a URL and a set of events in which you are
interested. For instance
https://developer.github.com/v3/activity/events/types/#pushevent[Github]
will POST to the webhook with a JSON body containing a list of
commits, and a header "X-Github-Event" equal to "push". If you add a
dependency on the `spring-cloud-config-monitor` library and activate
the Spring Cloud Bus in your Config Server, then a "/monitor" endpoint
is enabled.
When the webhook is activated the Config Server will send a
`RefreshRemoteApplicationEvent` targeted at the applications it thinks
might have changed. The change detection can be strategized, but by
default it just looks for changes in files that match the application
name (e.g. "foo.properties" is targeted at the "foo" application, and
"application.properties" is targeted at all applications). The strategy if you want to override the behaviour is `PropertyPathNotificationExtractor` which accepts the request headers and body as parameters and returns a list of file paths that changed.
The default configuration works out of the box with Github or
Gitlab. In addition to the JSON notifications from Github and Gitlab
you can trigger a change notification by POSTing to "/monitor" with a
form-encoded body parameters `path={name}`. This will broadcast to
applications matching the "{name}" pattern (can contain wildcards).
NOTE: the `RefreshRemoteApplicationEvent` will only be transmitted if
the `spring-cloud-bus` is activated in the Config Server and in the
client application.
NOTE: the default configuration also detects filesystem changes in
local git repositories (the webhook is not used in that case but as
soon as you edit a config file a refresh will be broadcast).
== Spring Cloud Config Client
A Spring Boot application can take immediate advantage of the Spring
View
@@ -22,13 +22,15 @@
</scm>
<properties>
<bintray.package>config</bintray.package>
<spring-cloud-bus.version>1.1.0.BUILD-SNAPSHOT</spring-cloud-bus.version>
<spring-cloud-commons.version>1.1.0.BUILD-SNAPSHOT</spring-cloud-commons.version>
</properties>
<modules>
<module>spring-cloud-config-client</module>
<module>spring-cloud-config-server</module>
<module>spring-cloud-config-monitor</module>
<module>spring-cloud-config-sample</module>
<module>spring-cloud-starter-config</module>
<module>spring-cloud-starter-config</module>
<module>docs</module>
</modules>
<dependencyManagement>
@@ -55,6 +57,11 @@
<artifactId>spring-cloud-config-server</artifactId>
<version>1.1.0.BUILD-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-config-monitor</artifactId>
<version>1.1.0.BUILD-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.springframework.retry</groupId>
<artifactId>spring-retry</artifactId>
@@ -0,0 +1,49 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-config</artifactId>
<version>1.1.0.BUILD-SNAPSHOT</version>
<relativePath>..</relativePath>
</parent>
<artifactId>spring-cloud-config-monitor</artifactId>
<name>spring-cloud-config-monitor</name>
<description>Spring Cloud Config Monitor</description>
<properties>
<main.basedir>${basedir}/../..</main.basedir>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-bus-parent</artifactId>
<version>${spring-cloud-bus.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-config-server</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-bus</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<!-- Only needed at compile time -->
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
</project>
@@ -0,0 +1,84 @@
/*
* Copyright 2015 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.config.monitor;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
import java.util.Map;
import org.springframework.core.Ordered;
import org.springframework.core.annotation.AnnotationAwareOrderComparator;
import org.springframework.core.annotation.Order;
import org.springframework.util.MultiValueMap;
/**
* A {@link PropertyPathNotificationExtractor} that cycles through a set of (ordered) delegates,
* looking for the first non-null outcome.
*
* @author Dave Syer
*
*/
public class CompositePropertyPathNotificationExtractor
implements PropertyPathNotificationExtractor {
private List<PropertyPathNotificationExtractor> extractors;
public CompositePropertyPathNotificationExtractor(
List<PropertyPathNotificationExtractor> extractors) {
this.extractors = new ArrayList<>();
if (extractors != null) {
this.extractors.addAll(extractors);
}
this.extractors.add(new SimplePropertyPathNotificationExtractor());
AnnotationAwareOrderComparator.sort(this.extractors);
}
@Override
public PropertyPathNotification extract(MultiValueMap<String, String> headers,
Map<String, Object> request) {
for (PropertyPathNotificationExtractor extractor : this.extractors) {
PropertyPathNotification result = extractor.extract(headers, request);
if (result != null) {
return result;
}
}
return null;
}
@Order(Ordered.LOWEST_PRECEDENCE - 200)
private static class SimplePropertyPathNotificationExtractor
implements PropertyPathNotificationExtractor {
@Override
public PropertyPathNotification extract(MultiValueMap<String, String> headers,
Map<String, Object> request) {
Object object = request.get("path");
if (object instanceof String) {
return new PropertyPathNotification((String) object);
}
if (object instanceof Collection) {
@SuppressWarnings("unchecked")
Collection<String> collection = (Collection<String>) object;
return new PropertyPathNotification(collection.toArray(new String[0]));
}
return null;
}
}
}
@@ -0,0 +1,43 @@
/*
* Copyright 2015 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.config.monitor;
import java.util.List;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.condition.ConditionalOnWebApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Import;
/**
* @author Dave Syer
*
*/
@Configuration
@ConditionalOnWebApplication
@Import(FileMonitorConfiguration.class)
public class EnvironmentMonitorAutoConfiguration {
@Autowired(required=false)
private List<PropertyPathNotificationExtractor> extractors;
@Bean
public PropertyPathEndpoint propertyPathEndpoint() {
return new PropertyPathEndpoint(new CompositePropertyPathNotificationExtractor(this.extractors));
}
}
Oops, something went wrong.

4 comments on commit 8d8e07a

@spencergibb

This comment has been minimized.

Show comment
Hide comment
@spencergibb

spencergibb Sep 24, 2015

Member

Nice! lot's of people asking for this.

Member

spencergibb replied Sep 24, 2015

Nice! lot's of people asking for this.

@sunels

This comment has been minimized.

Show comment
Hide comment
@sunels

sunels Apr 13, 2016

Let me try to fullfill this requirement then :)

Hi,

I will try to give all inputs firstly:

  1. ConfigServerApplication/application.yml
    spring:
    cloud:
    config:
    server:
    native:
    searchLocations: /space/projects/springConfigServer/configserver-master/src/main/localConfigRepo

  2. I am running the application as standalone jar : java -jar target/configserver-0.0.1-SNAPSHOT.jar --spring.profiles.active=native

  3. there is another application which is a client of configServer.For now it seems like to be get it's initial configuration from configserver.Actually it is a zuulProxy application and and i am trying to keep&fetch&refresh it's zuul.routes information the configServer.

  4. I am running both applications and changing the related application's config at the specified directory in the first step.

But while the Config Server is running up it says : o.s.c.c.m.FileMonitorConfiguration : Not monitoring for local config changes

I have read the below links but a bit confused about to build up a auto-refresh functioning CfgSrvClient- CfgServer application pair..maintaining spring dependencies is terrieble ..it makes me feel solving a crossword which is written kiril alphabet. And of course changing atributes does not take affect on the client side..

Maybe it is just bec.of dependency mess..

Do you have any idea what i am doing wrong ?

Kind regards

sunels replied Apr 13, 2016

Let me try to fullfill this requirement then :)

Hi,

I will try to give all inputs firstly:

  1. ConfigServerApplication/application.yml
    spring:
    cloud:
    config:
    server:
    native:
    searchLocations: /space/projects/springConfigServer/configserver-master/src/main/localConfigRepo

  2. I am running the application as standalone jar : java -jar target/configserver-0.0.1-SNAPSHOT.jar --spring.profiles.active=native

  3. there is another application which is a client of configServer.For now it seems like to be get it's initial configuration from configserver.Actually it is a zuulProxy application and and i am trying to keep&fetch&refresh it's zuul.routes information the configServer.

  4. I am running both applications and changing the related application's config at the specified directory in the first step.

But while the Config Server is running up it says : o.s.c.c.m.FileMonitorConfiguration : Not monitoring for local config changes

I have read the below links but a bit confused about to build up a auto-refresh functioning CfgSrvClient- CfgServer application pair..maintaining spring dependencies is terrieble ..it makes me feel solving a crossword which is written kiril alphabet. And of course changing atributes does not take affect on the client side..

Maybe it is just bec.of dependency mess..

Do you have any idea what i am doing wrong ?

Kind regards

@dsyer

This comment has been minimized.

Show comment
Hide comment
@dsyer

dsyer Apr 14, 2016

Contributor

No idea sorry (but you didn't mention using the bus anywhere - you need that to send the refresh messages). Perhaps a question on stack overflow would be better than a comment on an old commit?

Contributor

dsyer replied Apr 14, 2016

No idea sorry (but you didn't mention using the bus anywhere - you need that to send the refresh messages). Perhaps a question on stack overflow would be better than a comment on an old commit?

@sunels

This comment has been minimized.

Show comment
Hide comment
@sunels

sunels Apr 14, 2016

Thx..i am quite new to spring world..

But i think i have found the bug..

it is already explained by you..

I can see that migrating a boot app to a cloud app almost impossible for me.

I need to build it from strach from a sample cloud app step by step...

Hope to see you in stackoverflow..

Kind regards..

sunels replied Apr 14, 2016

Thx..i am quite new to spring world..

But i think i have found the bug..

it is already explained by you..

I can see that migrating a boot app to a cloud app almost impossible for me.

I need to build it from strach from a sample cloud app step by step...

Hope to see you in stackoverflow..

Kind regards..

Please sign in to comment.