SpringBoot 3.0.2 + Spring Cloud 2022.0.1 + Native / GraalVM / AOT: application.yml information not taken into account

[patpatpat123]

Hello Spring Cloud Stream team,

This is my first post in this repo, if not anything, many thanks for this project.
This project is very interesting, easy to use, easy to understand, the Youtube talks from Spring team are interesting to watch.

Just wanted to reach out regarding an issue with GraalVM / Spring Native / AOT please.
So far, it is 100% reproducible.

I have a piece of code, very straightforward, type "minimal reproducible example"

the code

package com.example.micrometer;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.messaging.Message;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import java.util.function.Consumer;

@SpringBootApplication
public class StreamReactiveConsumerApplication implements CommandLineRunner {

    private static final Logger log = LoggerFactory.getLogger(StreamReactiveConsumerApplication.class);

    public static void main(String... args) {
        new SpringApplicationBuilder(StreamReactiveConsumerApplication.class).run(args);
    }

    @Override
    public void run(String... args) {
    }

    @Bean
    Consumer<Flux<Message<String>>> consume() {
        return flux -> flux.flatMap(one -> myHandle(one) ).subscribe();
    }

    private Mono<String> myHandle(Message<String> one) {
        log.info("\u001B[32m" + one.getPayload() + "\u001B[0m");
        return Mono.just(one.getPayload().toUpperCase());
    }

}

the pom

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.example</groupId>
    <artifactId>streamreactiveconsumer</artifactId>
    <version>1.0-SNAPSHOT</version>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>3.0.2</version>
        <relativePath/>
    </parent>

    <properties>
        <maven.compiler.source>17</maven.compiler.source>
        <maven.compiler.target>17</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>2022.0.1</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-stream-binder-kafka</artifactId>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

the application.yml.

spring:
  application:
    name: stream-reactive-consumer
  cloud:
    stream:
      bindings:
        consume-in-0.destination: thetopic
      kafka:
        binder:
          brokers: mykafka.com:9092
          configuration:
            security:
              protocol: SSL
            ssl:
              keystore:
                location: keystore.p12
                password: abc
              truststore:
                location: truststore.p12
                password: abc

If I build this "the old fashion way" using mvn clean install, I get a jar, I run the jar, everything works fine.
I can see the consumer connecting the endpoint I mentioned, (int my example mykafka.com:9092) get the proper ssl certificate, etc.

2023-02-06 15:23:03 16592 INFO  --- [main] [stream-reactive-consumer,,] o.a.k.c.admin.AdminClientConfig : AdminClientConfig values: 
	bootstrap.servers = [mykafka.com:9092]
client.dns.lookup = use_all_dns_ips

So far so good.
Now, I have been trying to use the new AOT / GraalVM / Spring native.

I build the exact same project running mvn -Pnative native:compile, no code change at all. As a result, I do see the executable.

However, when I launch the executable, it is trying to connect to local host (despite me having the property set in application.yml)
Besides the host, it is like no other properties from application.yml are taken into account.

This is 100% reproducible with native image.

Could you please help on this issue?

Thank you

[sobychacko]

@patpatpat123 We are not able to reproduce the issue quickly. We created a sample that works all the way from regular JVM mode to native here. Could you go through these steps and make sure that it works for you? Please try to reproduce the issue that you are seeing against this app. That way, it is easier for us to triage it further.

[patpatpat123]

Thank you @sobychacko for the answer and the sample.
Those are very helpful, and, good news, I root caused the issue.

You are using:

spring:
  kafka:
    bootstrap-servers: the-kafka:9092

while I am using

spring:
  cloud:
    stream:
      kafka:
        binder:
          brokers: the-kafka:9092

(note: this property is a valid property found in Spring Cloud Stream documentation)

While using spring.kafka.bootstrap-servers does indeed workaround the problem, I do believe this might be an issue.
Not only this property, but also ssl properties such are keystores and truststores are impacted by this.
Please see below table for the bootstrap server property only, but this applies to many other configurations.

	classic application	native/aot/graal application
spring.kafka.*	OK	OK
spring.cloud.stream.kafka.*	OK	X

I believe there is an issue where, the spring.kafka.* properties would work for Spring Cloud Stream applications both native and classic.

However, spring.cloud.stream.kafka.* properties will not work if the application is built using native, but working when it is classic.

Is there a way to propose a registrar or a fix to keep working with please?

If this is expected, can there be some documentation on which spring cloud spring.cloud.stream.* properties are not supported when using native/aot/graalvm please?

Thank you

[sobychacko]

@patpatpat123 Thank you for the details. Indeed the spring.cloud.stream.kafka.* properties were broken on the native side. Our sample worked because, as you mentioned, we were using the spring.kafka.* properties. The spring.cloud.stream.kafka.* properties were only not working in the native mode; they were fine in the AOT mode. The issue is fixed now for native as well through the above-referenced commit and the fix should be available on the 4.0.2-SNAPSHOT version of the binder. Could you try your app with that version and see if it resolves your issue?

[patpatpat123]

Many thanks @sobychacko for the fix.

Indeed, it is working now.
In order to have this fix "live", as part of the spring-cloud-dependencies release, do I have to wait until 2022.0.2 or 2022.0.3?

[sobychacko]

It will be part of the 2022.0.2 release. It is scheduled to be released on 03/21. https://calendar.spring.io/.

[patpatpat123]

Marked on my calendar. Many thanks @sobychacko !