Best practices for Spring WebFlux + Spring Kafka after Reactor Kafka deprecation

[pikachom]

Hi Spring Kafka team,

First of all, thank you for all the work you put into this project.

I'm currently using Spring WebFlux together with reactor-kafka, and I'm trying to understand what the recommended pattern is now that Reactor Kafka and the reactive Kafka binder are being discontinued.

---

Previous setup (with Reactor Kafka)

Previously, my pipeline looked roughly like this (simplified):

KafkaReceiver<String, String> receiver = KafkaReceiver.create(receiverOptions);

receiver.receive() // Flux<ReceiverRecord<String, String>>
        .flatMap(this::doProcessRecord, 32) // bounded concurrency
        .subscribe();

Mono<Void> doProcessRecord(ReceiverRecord<String, String> record) {
    return findMetaFromDb(record)
            .flatMap(this::convert)
            .flatMap(this::publishConvertedRecord) // returns Mono<Void>
            .then(Mono.fromRunnable(() -> record.receiverOffset().acknowledge()));
}

This gave me:

• An end-to-end reactive pipeline (DB + HTTP, all based on Mono/Flux)
• At-least-once semantics (ack only after processing)
• A clear way to bound in-flight work via flatMap(..., concurrency) and Reactor Kafka’s backpressure.

Now that Reactor Kafka is being discontinued, I'm looking for the idiomatic way to do something similar using Spring Kafka only, but still staying “reactive” inside the processing.

---

Current idea: @KafkaListener returning Mono<Void>

Based on the reference docs for asynchronous @KafkaListener return types, my understanding is:

• A @KafkaListener method can return Mono<?> or CompletableFuture<?>.
• When an async return type is detected, the container automatically:
  • switches the AckMode to MANUAL, and
  • enables out-of-order commits / async acks,
  • and will ack the record when the Mono/future completes.
• Until all records from the previous poll are committed, the consumer is paused so no new records are delivered.

Given that, my current candidate implementation looks like this:

@KafkaListener(
    topics = "orders",
    groupId = "orders-group"
)
public Mono<Void> onMessage(ConsumerRecord<String, String> record) {
    return findMetaFromDb(record)
            .flatMap(this::convert)
            .flatMap(this::publishConvertedRecord) // Mono<Void>
            .then(); // Mono<Void>
}

Configuration-wise (simplified):

spring:
  kafka:
    consumer:
      group-id: orders-group
      enable-auto-commit: false
      max-poll-records: 32
      max-poll-interval-ms: 300000
    listener:
      concurrency: 1

My goal here is:

• No data loss / at-least-once semantics
• Reactive processing (WebFlux, R2DBC, WebClient, etc.) inside the listener
• Bounded in-flight work: effectively “up to 32 records in flight per consumer instance”, controlled by max-poll-records and concurrency.

---

Questions

1. Is this pattern (async @KafkaListener returning Mono<Void>) considered the recommended way to integrate Spring WebFlux–style reactive processing with Spring Kafka now that Reactor Kafka is discontinued?

   In other words, is this the preferred path going forward for reactive pipelines without depending on reactor-kafka?

2. For bounding in-flight work, is it correct to rely on:

   • max.poll.records × concurrency as the effective upper bound of in-flight records per application instance, and
   • the container’s automatic pausing (with async acks) to prevent the consumer from pulling more records until the previous poll’s records are all committed?

3. In a high-load / high-backlog scenario, would you recommend:

   • just tuning max.poll.records, concurrency, and max.poll.interval.ms along with the async listener,
   • or do you still recommend using explicit pause() / resume() from application code for more control when using WebFlux-heavy processing?

4. Are there any official samples, reference projects, or more detailed documentation that show an end-to-end pattern for “Spring WebFlux + Spring Kafka (without Reactor Kafka)” with at-least-once semantics and bounded concurrency?

   If not, would such a sample be something you’d be interested in accepting as a contribution?

---

I want to make sure I’m not over-engineering this, and that I’m aligning with the Spring Kafka team’s intended direction for reactive usage now that Reactor Kafka and the reactive binder are being phased out.

Any guidance or confirmation would be greatly appreciated.

Thanks!