-
Notifications
You must be signed in to change notification settings - Fork 13.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
KAFKA-13761: KafkaLog4jAppender deadlocks when idempotence is enabled #11939
Conversation
@@ -290,6 +291,7 @@ public void activateOptions() { | |||
props.put(DELIVERY_TIMEOUT_MS_CONFIG, deliveryTimeoutMs); | |||
props.put(LINGER_MS_CONFIG, lingerMs); | |||
props.put(BATCH_SIZE_CONFIG, batchSize); | |||
props.put(ENABLE_IDEMPOTENCE_CONFIG, false); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@yyu1993 Can you please add a comment explaining this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have added the comment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Thanks for the investigation and fix!
…#11939) When a log entry is appended to a Kafka topic using KafkaLog4jAppender, the producer.send operation may hit a deadlock if the producer network thread also tries to append a log at the same log level. This issue is triggered when idempotence is enabled for the KafkaLog4jAppender and the producer tries to acquire the TransactionManager lock. This is a temporary workaround to avoid deadlocks by disabling idempotence explicitly in KafkaLog4jAppender. Reviewers: Luke Chen <showuon@gmail.com>, Ismael Juma <ismael@juma.me.uk>
…#11939) When a log entry is appended to a Kafka topic using KafkaLog4jAppender, the producer.send operation may hit a deadlock if the producer network thread also tries to append a log at the same log level. This issue is triggered when idempotence is enabled for the KafkaLog4jAppender and the producer tries to acquire the TransactionManager lock. This is a temporary workaround to avoid deadlocks by disabling idempotence explicitly in KafkaLog4jAppender. Reviewers: Luke Chen <showuon@gmail.com>, Ismael Juma <ismael@juma.me.uk>
…#11939) When a log entry is appended to a Kafka topic using KafkaLog4jAppender, the producer.send operation may hit a deadlock if the producer network thread also tries to append a log at the same log level. This issue is triggered when idempotence is enabled for the KafkaLog4jAppender and the producer tries to acquire the TransactionManager lock. This is a temporary workaround to avoid deadlocks by disabling idempotence explicitly in KafkaLog4jAppender. Reviewers: Luke Chen <showuon@gmail.com>, Ismael Juma <ismael@juma.me.uk>
…apache#11939) When a log entry is appended to a Kafka topic using KafkaLog4jAppender, the producer.send operation may hit a deadlock if the producer network thread also tries to append a log at the same log level. This issue is triggered when idempotence is enabled for the KafkaLog4jAppender and the producer tries to acquire the TransactionManager lock. This is a temporary workaround to avoid deadlocks by disabling idempotence explicitly in KafkaLog4jAppender. Reviewers: Luke Chen <showuon@gmail.com>, Ismael Juma <ismael@juma.me.uk>
…cs-12-may-2022 * apache-kafka/3.0: (14 commits) fix: make sliding window works without grace period (#kafka-13739) (apache#11980) KAFKA-13794: Follow up to fix producer batch comparator (apache#12006) KAFKA-13794; Fix comparator of `inflightBatchesBySequence` in `TransactionManager` (apache#11991) KAFKA-13748: Do not include file stream connectors in Connect's CLASSPATH and plugin.path by default (apache#11908) KAFKA-13418: Support key updates with TLS 1.3 (apache#11966) KAFKA-13770: Restore compatibility with KafkaBasedLog using older Kafka brokers (apache#11946) KAFKA-13761: KafkaLog4jAppender deadlocks when idempotence is enabled (apache#11939) KAFKA-13759: Disable idempotence by default in producers instantiated by Connect (apache#11933) MINOR: Fix `ConsumerConfig.ISOLATION_LEVEL_DOC` (apache#11915) KAFKA-13750; Client Compatability KafkaTest uses invalid idempotency configs (apache#11909) ...
When a log entry is appended to a Kafka topic using KafkaLog4jAppender, the producer.send operation may hit a deadlock if the producer network thread also tries to append a log at the same log level. This issue is triggered when idempotence is enabled for the KafkaLog4jAppender and the producer tries to acquire the TransactionManager lock.
This is a temporary workaround to avoid deadlocks by disabling idempotence explicitly in KafkaLog4jAppender.
Committer Checklist (excluded from commit message)