Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Idempotency violation with modern kafka clients #3039

Closed
Tracked by #3038
rystsov opened this issue Nov 20, 2021 · 0 comments · Fixed by #3040
Closed
Tracked by #3038

Idempotency violation with modern kafka clients #3039

rystsov opened this issue Nov 20, 2021 · 0 comments · Fixed by #3040
Assignees
Labels
kind/bug Something isn't working

Comments

@rystsov
Copy link
Contributor

rystsov commented Nov 20, 2021

Version & Environment

Redpanda version: all versions up to 21.10.2

What went wrong?

Currently Redpanda doesn't do de duplication and returns OutOfOrderSequenceException when it suspects and an out of order record. Modern clients consider it as a non fatal error, bumps an epoch, resets seq number and retries the request. As a result in case of a network isolation (client <-> leader) or an leader termination (kill -9) a record may be produced twice.

What should have happened instead?

Redpanda should do de duplication or at least return a fatal error to force a user to close and recreate a producer

How to reproduce the issue?

https://github.com/vectorizedio/chaos

@rystsov rystsov added the kind/bug Something isn't working label Nov 20, 2021
@rystsov rystsov self-assigned this Nov 20, 2021
rystsov added a commit to rystsov/redpanda that referenced this issue Nov 20, 2021
Currently Redpanda doesn't do de duplication and returns OOOS error
when it suspects and an out of order record. Modern clients consider
it as a non fatal error, bumps an epoch, resets seq number and retries
the request. As a result in case of a network isolation (client <->
leader) or an leader termination (kill -9) a record may be produced
twice.

Mitigating the problem by returning a generic fatal error.

redpanda-data#3039
rystsov added a commit to rystsov/redpanda that referenced this issue Nov 20, 2021
Currently Redpanda doesn't do de duplication and returns OOOS error
when it suspects and an out of order record. Modern clients consider
it as a non fatal error, bumps an epoch, resets seq number and retries
the request. As a result in case of a network isolation (client <->
leader) or an leader termination (kill -9) a record may be produced
twice.

Mitigating the problem by returning a generic fatal error.

redpanda-data#3039
(cherry picked from commit 784554c)
travisdowns pushed a commit to travisdowns/redpanda that referenced this issue Nov 30, 2021
Currently Redpanda doesn't do de duplication and returns OOOS error
when it suspects and an out of order record. Modern clients consider
it as a non fatal error, bumps an epoch, resets seq number and retries
the request. As a result in case of a network isolation (client <->
leader) or an leader termination (kill -9) a record may be produced
twice.

Mitigating the problem by returning a generic fatal error.

redpanda-data#3039
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant