Skip to content

(chores) camel-kafka: improved the documentation for handling manual commits with camel-kafka #8769

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
davsclaus merged 1 commit into from
Nov 24, 2022
Merged

(chores) camel-kafka: improved the documentation for handling manual commits with camel-kafka #8769

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions components/camel-kafka/src/main/docs/kafka-component.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -369,13 +369,16 @@ This will force a synchronous commit which will block until the commit is acknow
You can use an asynchronous commit as well by configuring the `KafkaManualCommitFactory` with the `DefaultKafkaManualAsyncCommitFactory` implementation.

Then the commit will be done in the next consumer loop using the kafka asynchronous commit api.
Be aware that records from a partition must be processed and committed by a unique thread.
If not, this could lead with non-consistent behaviors.
This is mostly useful with aggregation's completion timeout strategies.

If you want to use a custom implementation of `KafkaManualCommit` then you can configure a custom `KafkaManualCommitFactory`
on the `KafkaComponent` that creates instances of your custom implementation.

*Note 1*: records from a partition must be processed and committed by the same thread as the consumer. This means that certain EIPs, async or concurrent operations
in the DSL, may cause the commit to fail. In such circumstances, tyring to commit the transaction will cause the Kafka client to throw a `java.util.ConcurrentModificationException`
exception with the message `KafkaConsumer is not safe for multi-threaded access`. To prevent this from happening, redesign your route to avoid those operations.

*Note 2: this is mostly useful with aggregation's completion timeout strategies.

== Pausable Consumers

The Kafka component supports pausable consumers. This type of consumer can pause consuming data based on
Expand Down