Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suppress EOF errors - only from logs #4704

Open
4 of 7 tasks
pushkarmoi opened this issue May 1, 2024 · 1 comment
Open
4 of 7 tasks

Suppress EOF errors - only from logs #4704

pushkarmoi opened this issue May 1, 2024 · 1 comment

Comments

@pushkarmoi
Copy link

Read the FAQ first: https://github.com/confluentinc/librdkafka/wiki/FAQ

Do NOT create issues for questions, use the discussion forum: https://github.com/confluentinc/librdkafka/discussions

Description

I'd like to treat EOF as an error (i.e. set enable.partition.eof to true) in my kafka consumer but not log it as one - is it doable?

ERROR rdkafka::client librdkafka: Global error: PartitionEOF

Logging configuration only allows me to change the default log level - which will suppress all ERROR level messages.
Can certain types of ERRORS be selectively suppressed?

How to reproduce

Open a consumer on an empty stream and poll

Checklist

IMPORTANT: We will close issues where the checklist has not been completed.

Please provide the following information:

  • librdkafka version (release number or git tag): 0.36.0
  • Apache Kafka version: 0.10.0
  • librdkafka client configuration: enable.partition.eof=true
  • Operating system: macOS
  • Provide logs (with debug=.. as necessary) from librdkafka
  • Provide broker log excerpts
  • Critical issue
@rolandyoung
Copy link

Are those version numbers for librdkafka and Apache Kafka correct? They seem very old indeed. Free support for librdkafka is generally available only for the current version. You should also note that the template you have filled in explicitly says "Do NOT create issues for questions, use the discussion forum" - you can probably expect to wait a long time to be answered by one of the project team.

Having said that, the answer to your question will depend on how you log errors in your application.

  • If you are using librdkafka directly, you will either be checking for an error after calling rd_kafka_consumer_poll or you will have implemented an error callback (error_cb); in either case you can modify your implementation to ignore this specific error. (See here for more details.)

  • If you are using a language binding (such as confluent-kafka-go or Perfect-Kafka), you will need to check the documentation for that binding or seek support from its developers.

(Note: I am not a member of the librdkafka project, I am answering as a fellow user of librdkafka)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants