Log http request headers - like "correlationId"

[t2gran]

Summary - "traceParameters"

Trace OTP request using HTTP request/response parameter(s) combined with logging.

OTP support tracing user requests across log events and "outside" services. OTP can insert
http-request-header parameters into all associated log events and into the http response. If the
value is not present in the request, a unique value can be generated. The otp generated value is
6 characters long base 36[0-9a-z] character string.

Correlation-ID example

A common use-case in a service oriented environment is to use a correlation id to identify all log
messages across multiple services from the same user. This can be done by setting the "X-Correlation-ID" http header in the http facade/gateway. To pick up the correlation Id from the header, inset it in all relevant log messages and to return it in the http response header configure OTP like this:

"traceParameters": [
  {
    "httpRequestHeader": "X-Correlation-ID",
    "httpResponseHeader": "X-Correlation-ID",
    "mdcLogKey": "correlationId",
    "generateIdIfMissing": true
  }
]

Now, to output the correlationId in the log using logback you can include %X{correlationId} in the log pattern:

<pattern>%d{HH:mm:ss.SSS} %level [%thread] %X{correlationId} \(%F:%L\) %msg%n</pattern>

Unit tests

✅

Documentation

✅

Changelog

✅

Bumping the serialization version id

✅

[codecov]

Codecov Report

Patch coverage: 25.95% and project coverage change: +0.02 🎉

Comparison is base (c07adcd) 65.07% compared to head (85557aa) 65.09%.

Additional details and impacted files

@@              Coverage Diff              @@
##             dev-2.x    #5131      +/-   ##
=============================================
+ Coverage      65.07%   65.09%   +0.02%     
- Complexity     14244    14338      +94     
=============================================
  Files           1733     1746      +13     
  Lines          67557    67852     +295     
  Branches        7203     7235      +32     
=============================================
+ Hits           43963    44170     +207     
- Misses         21155    21236      +81     
- Partials        2439     2446       +7     

Impacted Files	Coverage Δ
...ipplanner/framework/application/LogMDCSupport.java	0.00% <0.00%> (ø)
.../framework/concurrent/LogMDCRunnableDecorator.java	0.00% <0.00%> (ø)
.../framework/concurrent/OtpRequestThreadFactory.java	62.50% <0.00%> (-20.84%)	⬇️
...ner/standalone/configure/ConstructApplication.java	19.14% <0.00%> (-1.31%)	⬇️
...ipplanner/standalone/server/OTPWebApplication.java	0.00% <0.00%> (ø)
...standalone/server/OTPWebApplicationParameters.java	0.00% <0.00%> (ø)
...pplanner/standalone/server/RequestTraceFilter.java	9.09% <9.09%> (ø)
...anner/standalone/server/RequestTraceParameter.java	44.44% <44.44%> (ø)
...r/standalone/config/routerconfig/ServerConfig.java	100.00% <100.00%> (ø)

... and 30 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[vpaturet]

should we sanitize the header value?

[t2gran]

What would be a good pattern/method to apply ? [^\c\s]{1,1000} (1 to 1000 characters and NOT control or space)?

[t2gran]

Looked into it, and you are allowed to pass in a lot, so I think we should not validate this. If we should then we should add a regular expression to the config for the trace parameter.

[vpaturet]

I would say yes. This is sufficient to prevent log forging. Preventing all forms of script injections is probably out of scope here.

[leonardehrenfried]

This looks good but I picked up on a few spelling mistakes.