Kafka Client Compatibility

Sabby Anandan edited this page Aug 19, 2018 · 6 revisions

Spring Cloud Stream's Kafka binder builds upon Spring Kafka and Spring Integration Kafka libraries. Specifically, Spring Kafka is the core to this foundation, and it is based on the pure java kafka-clients jar.

The following is the compatibility matrix of Spring cloud Stream and its dependent projects (and broker versions):

Spring Cloud Stream Version Spring for Apache Kafka Version Spring Integration for Apache Kafka Version kafka-clients Kafka Broker Version
2.0.x 2.1.x 3.0.x 1.1.x, 1.0.x(*) 1.1.x, 1.0.x, 0.11.0.x(*)
1.3.x 1.3.x, 1.2.x, 1.1.x 2.3.x, 2.2.x, 2.1.x 0.11.0.x(**), 0.10.2.x 0.10.x.x or higher
1.2.x   1.2.x, 1.1.x   2.2.x, 2.1.x    0.10.1.x    0.10.x.x or higher

Depending on the selected release-combination, the dependent projects will be referenced transitively when using maven or gradle for version management.

Notes:

  • The 1.3.x supports using newer clients (e.g. 1.x, 2.x) but automatic provisioning is not supported; the topic(s) must already exist.
  • Older versions of Spring Boot (<= 1.5) pull in spring-kafka 1.1.x by default; all users are recommended to override the spring-kafka version to the latest 1.3.x version.
  • (*) The 2.0.x Kafka-binder uses the pure java AdminClient to provision topics and supports native headers.
  • The 1.x.x client can communicate with a 0.11.x.x broker (which also supports headers).

Kafka 0.11 Notes:

  • Important The Kafka11 binder uses native headers exclusively, it is not compatible with brokers older than 0.11, nor is it compatible with applications that use the standard 1.3.x kafka binder configured to use embedded headers.
  • When using with a 0.11.x.x broker, topics can be provisioned, but the number of partitions cannot be automatically adjusted up. To increase the number of topics, you must use the kafka tools instead. The property spring.cloud.stream.kafka.binder.auto-add-partitions must be false (default).
  • (**) To use the 0.11.x.x kafka-clients with 1.3.x, you can use the spring-cloud-stream-binder-kafka11 jar (instead of spring-cloud-stream-binder-kafka) to get support for native headers. You must also override certain other jar versions as follows:
Dependency Version
spring-cloud-stream-binder-kafka11 1.3.0.RELEASE or higher
spring-kafka 1.3.x, 1.2.x, 1.1.x
spring-integration-kafka   2.3.0.RELEASE or higher  
spring-integration-core   4.3.13.RELEASE or higher  
spring-integration-jmx   4.3.13.RELEASE or higher  
kafka_2.11   (same version as kafka-clients)
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.