Not seeing MDC values flow into customDimensions when using the App Insights Java Agent (preview 5)

[andrejpk]

Expected behavior

Any key/value pairs set in the MDC (SLF4J) should appear in App Insights' customDimensions collection. This worked before we enabled the App Insights agent. This is in a Spring Boot application.

Actual behavior

customDimensions contains some standard data like LoggerName and LoggingLevel but nothing from MDC carries over. I can confirm locally in the console logs that these MDC values are being set and emitted to console logs (via my console log custom format strings).

To Reproduce

Steps to reproduce the behavior:

pom xml dependencies:

<dependencies>
		<!-- General/Spring -->	
        <dependency>
            <groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
			<version>2.3.2.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
			<version>2.3.2.RELEASE</version>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>org.junit.vintage</groupId>
                    <artifactId>junit-vintage-engine</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <!-- Logging/Instrumentation -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-actuator</artifactId>
			<version>2.3.2.RELEASE</version>
        </dependency>
		<dependency>
			<groupId>com.microsoft.azure</groupId>
			<artifactId>applicationinsights-spring-boot-starter</artifactId>
			<version>2.6.1</version>
		</dependency>
        <dependency>
            <groupId>com.microsoft.azure</groupId>
            <artifactId>applicationinsights-logging-logback</artifactId>
            <version>2.6.1</version>
        </dependency>
        <dependency>
            <groupId>com.microsoft.azure</groupId>
            <artifactId>azure-spring-boot-metrics-starter</artifactId>
            <version>2.3.2</version>
        </dependency>
        <!-- Security -->
        <dependency>
            <groupId>org.springframework.security</groupId>
            <artifactId>spring-security-oauth2-resource-server</artifactId>
            <version>5.3.3.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.security</groupId>
            <artifactId>spring-security-oauth2-jose</artifactId>
            <version>5.3.3.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.security</groupId>
            <artifactId>spring-security-config</artifactId>
            <version>5.3.3.RELEASE</version>
        </dependency>
        <!-- Config -->
		<dependency>
			<groupId>org.springframework.cloud</groupId>
			<artifactId>spring-cloud-config-client</artifactId>
			<version>2.2.4.RELEASE</version>
		</dependency>
    </dependencies>

Running AI Agent 3.0 Preview 5

pre-request handler sets MDCs:

	@Override
	public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {
		String requestIdHeader = request.getHeader(RequestIdHeaderName);
		if (requestIdHeader == null) {
			requestIdHeader = UUID.randomUUID().toString();
		}
		String correlationIdHeader = request.getHeader(CorrelationIdHeaderName);
		if (correlationIdHeader == null) {
			correlationIdHeader = UUID.randomUUID().toString();
		}
		// store logging context on this thread so all subsequent logs get the
		// correlation and request IDs
		MDC.put(LogRequestKey, requestIdHeader);
		MDC.put(LogCorrelationKey, correlationIdHeader);
		logger.info("Handling HTTP request: " + LogRequestKey + ":" + requestIdHeader + " " + LogCorrelationKey + ":"
				+ correlationIdHeader + "Reading back getCorrelationID(): " + getCorrelationID()) ;
		return true;
	}
	public static String getCorrelationID() {
		return MDC.get(LogCorrelationKey);
	}

In the AI trace logs:

message: Handling HTTP request: RequestID:6c315816-544c-4b13-87e2-0da470260b5b CorrelationID:64532ae4-b89c-4755-a96b-985cd52952f9Reading back getCorrelationID(): 64532ae4-b89c-4755-a96b-985cd52952f9
customDimensions: {"LoggerName":"com.xxxxxxxx.logging.LoggerInterceptor","LoggingLevel":"INFO","SourceType":"Logger","TimeStamp":"Fri, 14 Aug 2020 13:56:54 GMT"}

Sample Application

code blocks above

System information

Please provide the following information:

• SDK Version: full list above in pom.xml
• OS type and version:
• Application Server type and version (if applicable):
• Using spring-boot?
• Additional relevant libraries (with version, if applicable):

Logs

n/a

Screenshots

[MS-jgol]

@andrejpk This will be supported in the GA version that will be released in early November

[trask]

Preview.7 now captures MDC values as customDimensions

[SaurabhWajpe]

We can now see the MDC values as customDimensions, but We have a weird issue with Preview 7 version.

We have a microservice deployed on AKS and during development we usually connect to bastion using SSH tunnel. We also have APIM configured to point to same service.

With Preview 7 version, we can see all the logs when we hit to SSH tunnel url but none when we hit to APIM endpoint.

We tested the same scenario with Preview 5 version and apart from MDC values, we can see all the logs.

Any pointers?