Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Broker version compatibility
librdkafka supports all released Apache Kafka broker versions since 0.8.0.0.0, but not all features may be available on all broker versions since some features rely on newer broker functionality.
api.version.request=true(CHANGED to true in librdkafka v0.11.0)
Depending on what broker version you are using, please configure your librdkafka based client as follows:
Broker version >= 0.10.0.0 (or trunk)
(this is the default value)
Broker versions 0.9.0.x
api.version.request=false broker.version.fallback=0.9.0.x (the exact 0.9.0.. version you are using)
Broker versions 0.8.x.y
api.version.request=false broker.version.fallback=0.8.x.y (your exact 0.8... broker version)
Apache Kafka version 0.10.0.0 adds support for KIP-35 - querying the broker for supported API request types and versions - allowing the client to figure out what features it can use. But for older broker versions there is no way for the client to reliably know what the broker supports.
To alleviate this situation librdkafka adds three new configuration properties:
api.version.request=true|false- enables the API version request, this requires a >= 0.10.0.0 broker and will cause a disconnect on brokers 0.8.x - this disconnect is recognized by librdkafka and on the next connection attempt (which is immediate) it will disable the API version request and use
broker.version.fallbackas a basis of available features. NOTE: Due to a bug in broker version 0.9.0.0 & 0.9.0.1 the broker will not close the connection when receiving the API version request, instead the request will time out in librdkafka after 10 seconds and it will fall back to
broker.version.fallbackon the next immediate connection attempt.
broker.version.fallback=X.Y.Z.N- if the API version request fails (if
api.version.request=true) or API version requests are disabled (
api.version.request=false) then this tells librdkafka what version the broker is running and adapts its feature set accordingly.
api.version.fallback.ms=MS- In the case where
api.version.request=trueand the API version request fails, this property dictates for how long librdkafka will use
MShas passed the API version request will be sent on any new connections made for the broker in question. This allows upgrading the Kafka broker to a new version with extended feature set without needing to restart or reconfigure the client (given that
Note: These properties applies per broker.
The API version query was disabled by default (
api.version.request=false) in librdkafka up to and including v0.9.5 due to the afforementioned bug in broker version 0.9.0.0 & 0.9.0.1, but was changed to
true in librdkafka v0.11.0.