throw out error message "Local: Message timed out

[ai0376]

Description

create topic use this shell:
bin/kafka-topics.sh --create --zookeeper localhost:3181 --replication-factor 3 --partitions 1 --topic hadoop148_cin_im_SCS01

create 20 topic , and start 5 client and connect kafka，

METADATA|rdkafka#producer-1|192.168.2.21:19092/2: Metadata request failed: Local: Message timed out

what‘s wrong？

How to reproduce

Checklist

Please provide the following information:

• librdkafka version (release number or git tag): 0.8
• Apache Kafka version: kafka_2.11-0.8.2.2
• librdkafka client configuration:
• Operating system: centos 6.5 x64
• Using the legacy Consumer
• Using the high-level KafkaConsumer
• Provide logs (with debug=.. as necessary) from librdkafka
• Provide broker log excerpts
• Critical issue

[edenhill]

Can you try librdkafka 0.9.1 or latest master? 0.8.??? is very old.
Also: what client are you creating? Producer of consumer?

[ai0376]

review librdkafka source code, "Local: Message timed out" string is defined RD_KAFKA_RESP_ERR__MSG_TIMED_OUT, /* Produced message timed out/

[ai0376]

my produce config this:
`
100000000

  <opt name="metadata.request.timeout.ms">60000</opt>

  <opt name="metadata.refresh.interval.ms">10000</opt>

  <opt name="metadata.refresh.fast.cnt">10</opt>

  <opt name="metadata.refresh.fast.interval.ms">200</opt>

  <opt name="debug">msg</opt> 

  <opt name="socket.timeout.ms">100</opt>

  <opt name="socket.send.buffer.bytes">4096000</opt>

  <opt name="socket.receive.buffer.bytes">4096000</opt>

  <opt name="broker.address.ttl">3600000</opt>

  <opt name="queue.buffering.max.messages">100000</opt>

  <opt name="queue.buffering.max.ms">100</opt>

  <opt name="batch.num.messages">1000</opt>

  <opt name="message.send.max.retries">2</opt>

  <opt name="retry.backoff.ms">100</opt>

  <opt name="compression.codec">gzip</opt> 

create 40 topic on 1 partitions.

Is my topic too many ? I try to change socket.timeout.ms = 3000 ,but it is not effect!

[edenhill]

40 topics should not be a problem

Can you try librdkafka 0.9.1 or latest master?

[edenhill]

socket.timeout.ms is the request timeout, it should allow for:
local queueing time + network RTT latency + broker processing time

Setting it to 100ms is very low (it defaults to 60s), what is your reason for this low value?

[edenhill]

This happens when the broker latency is high.
A fix that improves error handling in these scenarios will be available on master soon

[edenhill]

This issue is now fixed on master, please try to verify the fix in your environment.
Thank you.

[ai0376]

Thanks， i will try it! Best wishes！