Creek Kafka

Kafka integration for Creek.

An example of how to write Kafka Streams based microservices using this Creek extension can be found in the aptly named basic Kafka Streams tutorial.

See CreekService.org for more info on using this extension.

Modules defined in this repo:

Published:

• metadata [JavaDocs]: defines types to allow your services to indicate they consume and produce to Kafka.
• client-extension [JavaDocs]: defines a Creek extension to help with writing Kafka client based microservices.
• streams-extension [JavaDocs]: defines a Creek extension to help with writing Kafka Streams based microservices.
• serde [JavaDocs]: defines the base types used to define Kafka Serde (serializers and deserializers), and a kafka serialization format which uses the standard Kafka client serde.
• serde-test [JavaDocs]: Test utils for testing serde
• json-serde [JavaDocs]: provides JSON serde implementations.
• streams-test [JavaDocs]: Helpers for writing tests for Kafka streams based microservices.
• test-extension [JavaDocs]: Creek system-test extension to allow system testing of Kafka based microservices.

Internal / Non-published:

• test-java-eight: functional tests without Java 9's modularity.
• test-java-nine: functional tests with Java 9's modularity.
• test-serde: test-only serde extension implementation.
• test-serde-java-eight: test-only serde extension implementation that only registers itself using the Java 8 META-INF/services method.
• test-service-native: test-only microservice implementation using inbuilt/native Kafka serde.
• test-service-json: test-only microservice implementation using json-serde.

Docs:

• docs: doc site accessible on https://www.creekservice.org/creek-kafka.
• docs-examples: example code injected into the docs.

Kafka client version compatability

The libraries themselves are compiled with the latest versions of the Kafka clients and Kafka streams libraries. However, they are compatible with older versions, as set out in the table below. The tested version column details the exact version of Kafka libraries testing covers.

Kafka version	Tested version	Notes
< 2.8		Not compatible due to API changes in Streams
2.8.+	2.8.2	Supported & tested
3.0.+	3.0.2	Supported & tested
3.1.+	3.1.2	Supported & tested
3.2.+	3.2.3	Supported & tested
3.3.+	3.3.2	Supported & tested
3.4.+	3.4.1	Supported & tested
3.5.+	3.5.2	Supported & tested
3.6.+	3.6.1	Supported & tested
> 3.6		Not currently tested / released. Should work...

In Gradle, it is possible to force the use of an older Kafka client if you wish using a resolution strategy:

configurations.all {
    resolutionStrategy.eachDependency {
        if (requested.group == "org.apache.kafka") {
            useVersion("2.8.2")
        }
    }
}