@RetryableTopic not working for asynchronous @KafkaListener return types

[matthiasValuecloud]

In what version(s) of Spring for Apache Kafka are you seeing this issue?

3.2.0

Describe the bug

When using the @RetryableTopic annotation on a Listener that returns a CompletableFuture, no retry or dlt events are sent when the CompletableFuture completes with an exception.

To Reproduce

@RetryableTopic(
            attempts = "5",
            kafkaTemplate = "retryTemplate",
            autoCreateTopics = "false",
            backoff = @Backoff(delay = 6000)
    )
@KafkaListener(
        topics = "my-test-topic",
        properties = {
                "key.deserializer=io.confluent.kafka.serializers.KafkaAvroDeserializer",
                "value.deserializer=io.confluent.kafka.serializers.KafkaAvroDeserializer",
                "specific.avro.reader=true"
        },
        id = "myId"
)
public CompletableFuture<Void> receive(ConsumerRecord<Key, Value> record) {
    log.info("Received request (id {}) on topic {}", record.key().getId(), record.topic());

    CompletableFuture<Void> result = processor.process(record.key(), record.value());
    result.whenComplete((v, ex) -> {
       if (ex != null) {
           log.error("Error processing request: {}", ex.getMessage());
       }
    });
    return result;
}

Expected behavior

A CompletableFuture, which results in an exception, should be retried with the given properties and marked for ack on my-topic.

[sobychacko]

@matthiasValuecloud This is not a bug, but something we never added as a feature for non-blocking retries with async KafkaListener methods. We can try to make it in before the next releases, but any PR contributions are welcome. Thanks!

[matthiasValuecloud]

Well i can try, but i am really not comfortable around such a huge project and how to contribute properly. I will read through the guidelines and see what i can do :D

[sobychacko]

@matthiasValuecloud Sounds good. Please let us know if you end up working on it. Here are the contribution guidelines: https://github.com/spring-projects/spring-kafka/blob/main/CONTRIBUTING.adoc.

[matthiasValuecloud]

Haven't had the chance on working on it so far. I apologize.

[artembilan]

Moving to Backlog since we haven't have a chance to look into this.

[chickenchickenlove]

@matthiasValuecloud Hi!
If you don't mind, may i take a look this issue?

or @sobychacko , @artembilan
Would it be alright if i tried to solve this issue?

[artembilan]

Be our guest, @chickenchickenlove !
We have some other priorities to dig out, but will more than happy to review a possible fix or discuss some specific details .
Thank you for looking into this!

[matthiasValuecloud]

@chickenchickenlove thank you! I would really like to appreciate if you take over.. i had only a couple of hours to look into it and had no success so far. I hope you have more success then me :)