Spring boot support for correlation IDs

[kelapure]

End-to-End request flow measurements are very important. A Correlation ID is used across request calls to track microservice call flow to 1. nail down nasty bugs and 2. track the originating event and understand the latency between the edge service to the API Gateway and from the API gateway fanout to all the downstream services. This is achieved by inserting correlation GUIDs and transaction IDs in HTTP headers and tracking them across requests.

• Correlation IDs can be implemented as servlet filters. A better way to inject this cross cutting concern across microservices code is by defining an aspect. Building on the servlet filter approach, such a capability is provided by the com.ofg:micro-infra-spring-base module. The CorrelationID support in the 4finance repo. works across Spring Reactor threads, Servlet 3.0, async servlets, Spring MVC Controllers, Spring RestTemplate and Apache Camel.
• The finagle-zipkin project provides the instrumentation for google’s Dapper-style distributed tracing in services. See Zipkin for a full fledged tracing system.
• Cloud Foundry services like AppDynamics and NewRelic also provide this capability by automatically instrumenting the java code via javaagent’s to insert these unique IDs and then visualize the request flow using a dashboard.

Can we get the capability of leveraging correlation IDs to enable request tracing across process boundaries natively in spring boot.

[kelapure]

To give you a concrete use case ...

The Cloud Foundry PaaS sets the following environment headers on every application

VCAP_APPLICATION={"instance_id":"451f045fd16427bb99c895a2649b7b2a",
"instance_index":0,"host":"0.0.0.0","port":61857,"started_at":"2013-08-12
00:05:29 +0000","started_at_timestamp":1376265929,"start":"2013-08-12 00:05:29
+0000","state_timestamp":1376265929,"limits":{"mem":512,"disk":1024,"fds":16384}
,"application_version":"c1063c1c-40b9-434e-a797-db240b587d32","application_name"
:"styx-james","application_uris":["styx-james.a1-app.cf-app.com"],"version":"c10
63c1c-40b9-434e-a797-db240b587d32","name":"styx-james","uris":["styx-james.a1-ap
p.cf-app.com"],"users":null}

The Cloud Foundry PaaS sets the following Request Headers on every request

• X-CF-InstanceID
• X-CF-ApplicationID
• X-Vcap-Request-Id

Therefore the VCAP_APPLICATION environment variable (application_name, application_version) + Request headers have everything you need to create an application request correlation ID that enables request correlation across JVM process boundaries. You correlate app name via X-CF-ApplicationID + VCAP_APPLICATION: application_version. In a microservices architecture where one request could potentially traverse multiple JVMs this is very useful.

The application name stays the same across pushes of an app, or follow some convention (like appname_v123). So the logging system could use a rule like startswith or split the name by _. This enables long term trend analysis of the logs across multiple app pushes.

ergo this feature request is for the insertion of a configurable correlation ID for spring boot application logs

[philwebb]

Thanks for the detailed info. I'll try to target this for 1.3.

[spencergibb]

@philwebb this is also something on spring-cloud's radar. See this project for some auto-config support for a zipkin compatible java library https://github.com/kristofa/brave. I have support for RestTemplate. Was going to work on integrating a servlet filter and a zuul filter.

[philwebb]

@spencergibb I actually think this might be a better fit as a Spring Cloud project. Perhaps we just need to make sure that there are appropriate hook points in Spring Boot 1.3.

[philwebb]

This will be handled in Spring Cloud.

[wbauern]

Does this have a corresponding issue in the Spring Cloud project now? I just did a search and couldn't find anything for 'correlation'. We need this functionality and are trying to roll our own right now. I want to track it to see when we can cut over to the Spring Cloud implementation. Thanks!

[shakuzen]

I would also be very interested in using this feature and hope it can be included in the next release of Spring Cloud and Spring Boot.

[spencergibb]

spring-cloud/spring-cloud-commons#11