LoggerContext changes when using Spring Cloud, eliminating Logback metrics

[jkschneider]

See micrometer-metrics/micrometer#564.

It seems from the OP's description that every time the context is refreshed, the Logback turbo filter list is reset. His workaround is to rebind Logback metrics on each event in a ApplicationListener<ContextRefreshedEvent>.

Any thoughts?

[jkschneider]

There's also a potentially related issue in sc-netflix: spring-cloud/spring-cloud-netflix#2830.

cc / @spencergibb

[spencergibb]

Doesn't even need refresh with spring cloud commons, just having spring cloud stream does it spring-cloud/spring-cloud-stream#1355

[jkschneider]

Yep, that seems to narrow the issue down to LoggingApplicationListener. Unfortunately we can't bind LogbackMetrics somewhere in there since that is in the Spring Boot core module.

[philwebb]

Is there any code to that replicates this issue?

[jkschneider]

@philwebb A reproducible example is included in spring-cloud/spring-cloud-stream#1355

[vprudnikov]

Any results on this? I followed all the related issues and all of them are closed so this is probably the one which is opened.

Anyway I would like to add some details that can be helpful:

A spring cloud application with Spring Boot 2.1.3.RELEASE

I found out that first, io.micrometer.core.instrument.binder.logging.LogbackMetrics registers a logback filter which should work fine:

LogbackMetrics
...
public void bindTo(MeterRegistry registry) {
    MetricsTurboFilter filter = new MetricsTurboFilter(registry, tags);
    metricsTurboFilters.put(registry, filter);
    loggerContext.addTurboFilter(filter); // <- this is the line
}

But then, org.springframework.boot.context.logging.LoggingApplicationListener receives a ApplicationEnvironmentPreparedEvent and re-initializes logging system:

LogbackLoggingSystem
...
public void initialize(LoggingInitializationContext initializationContext,
      String configLocation, LogFile logFile) {
   LoggerContext loggerContext = getLoggerContext();
   if (isAlreadyInitialized(loggerContext)) { // <- this returns FALSE
      return;
   }
   super.initialize(initializationContext, configLocation, logFile); // <- this is the line!
   loggerContext.getTurboFilterList().remove(FILTER);
.......

and LogbackLoggingSystem calls 'stopAndReset':

LogbackLoggingSystem
...
protected void loadDefaults(LoggingInitializationContext initializationContext,
      LogFile logFile) {
   LoggerContext context = getLoggerContext();
   stopAndReset(context); // <- this line leads to LoggerContext#resetTurboFilterList method invocation
...

After that, the metrics filter is removed from the LoggerContext so metrics cannot be collected.