Skip to content

@KafkaListener with Both topics and topicPartitions Breaks @RetryableTopic #4170

@moonyoungCHAE

Description

@moonyoungCHAE

In what version(s) of Spring for Apache Kafka are you seeing this issue?

3.3.7

Describe the bug

  1. When I declare a consumer using @KafkaListener and mix topics and topicPartitions, no error occurs. The consumer ends up consuming messages from the topics specified in topicPartitions. I think it would be more user-friendly if a validation exception were thrown in this case. Would it make sense to add such validation?

  2. When both topics and topicPartitions are declared in @KafkaListener and the @RetryableTopic annotation is used, the retry topic is managed based on topics. However, as explained in point 1, the consumer actually receives messages from the topics in topicPartitions. As a result, if an error occurs while processing messages from topicPartitions, the retry mechanism does not work correctly.

To Reproduce

  1. Create a consumer with @KafkaListener that uses both topics and topicPartitions.
  2. Add the @RetryableTopic annotation.
  3. Send a message to a topicPartition and trigger an exception in the consume

Expected behavior

Sample

public class MyListener {
    @RetryableTopic(attempts = "3", backoff = @Backoff(delay = 1000, multiplier = 2), topicSuffixingStrategy = TopicSuffixingStrategy.SUFFIX_WITH_INDEX_VALUE)
    @KafkaListener(id = "foo",
            topics = "myTopic1",
            clientIdPrefix = "myClientId", topicPartitions =
            {
                    @TopicPartition(topic = "topic1", partitions = {"0"}),
                    @TopicPartition(topic = "topic2", partitions = {"0"})
            })
    public void listen(String data) {
        throw new RuntimeException("a");
    }
}
2025-11-17T18:32:05.694+09:00 ERROR 40508 --- [      foo-0-C-1] o.s.kafka.listener.DefaultErrorHandler   : Failed to determine if this record (topic1-0@9) should be recovered, including in seeks

java.lang.NullPointerException: No destination found for topic: topic1
	at java.base/java.util.Objects.requireNonNull(Objects.java:340) ~[na:na]
	at org.springframework.kafka.retrytopic.DefaultDestinationTopicResolver.doGetDestinationFor(DefaultDestinationTopicResolver.java:230) ~[spring-kafka-3.3.7.jar:3.3.7]
	at org.springframework.kafka.retrytopic.DefaultDestinationTopicResolver.getDestinationTopicSynchronized(DefaultDestinationTopicResolver.java:220) ~[spring-kafka-3.3.7.jar:3.3.7]
	at org.springframework.kafka.retrytopic.DefaultDestinationTopicResolver.getDestinationHolderFor(DefaultDestinationTopicResolver.java:214) ~[spring-kafka-3.3.7.jar:3.3.7]
	at org.springframework.kafka.retrytopic.DefaultDestinationTopicResolver.resolveDestinationTopic(DefaultDestinationTopicResolver.java:99) ~[spring-kafka-3.3.7.jar:3.3.7]
	at org.springframework.kafka.retrytopic.DeadLetterPublishingRecovererFactory.lambda$destinationResolver$11(DeadLetterPublishingRecovererFactory.java:289) ~[spring-kafka-3.3.7.jar:3.3.7]
	at org.springframework.kafka.listener.DeadLetterPublishingRecoverer.accept(DeadLetterPublishingRecoverer.java:504) ~[spring-kafka-3.3.7.jar:3.3.7]
	at org.springframework.kafka.listener.FailedRecordTracker.attemptRecovery(FailedRecordTracker.java:228) ~[spring-kafka-3.3.7.jar:3.3.7]

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions