Skip to content
A collection of Single Message Transformations (SMTs) for Kafka Connect
Java
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
config/checkstyle Add integration tests Oct 2, 2019
gradle/wrapper Do initial code and README check-in Sep 5, 2019
src
.gitignore Do initial code and README check-in Sep 5, 2019
.travis.yml Add integration tests Oct 2, 2019
LICENSE Initial commit Sep 5, 2019
README.md
build.gradle Add integration tests Oct 2, 2019
gradle.properties
gradlew
gradlew.bat Do initial code and README check-in Sep 5, 2019
settings.gradle

README.md

Aiven Kafka Connect Transformations

Build Status

This is a set of Kafka Connect transformations.

Transformations

See the Kafka documentation for more details about configuring transformations.

ExtractTimestamp

This transformation replaces the original record's timestamp with a value taken from the the record.

The transformation:

  • expects the record value to be either a STRUCT or a MAP;
  • expects it to have a specified field;
  • expects the value of the field to be either INT64 or org.apache.kafka.connect.data.Timestamp and not be null.

Exists in one variant:

  • io.aiven.kafka.connect.transforms.ExtractTimestamp$Value - works on values.

The transformation defines the following configurations:

  • field.name - The name of the field which should be used as the new timestamp. Cannot be null or empty.

Here's an example of this transformation configuration:

transforms=ExtractTimestampFromValueField
transforms.ExtractTimestampFromValueField.field.name=inner_field_name

ExtractTopic

This transformation extracts a string value from the record and use it as the topic name.

The transformation can use either the whole key or value (in this case, it must have STRING type) or a field in them (in this case, it must have STRUCT type and the field's value must be STRING).

Exists in two variants:

  • io.aiven.kafka.connect.transforms.ExtractTopic$Key - works on keys;
  • io.aiven.kafka.connect.transforms.ExtractTopic$Value - works on values.

The transformation defines the following configurations:

  • field.name - The name of the field which should be used as the topic name. If null or empty, the entire key or value is used (and assumed to be a string). By default is null.
  • skip.missing.or.null - In case the source of the new topic name is null or missing, should a record be silently passed without transformation. By default is false.

Here's an example of this transformation configuration:

transforms=ExtractTopicFromValueField
transforms.ExtractTopicFromValueField.type=io.aiven.kafka.connect.transforms.ExtractTopic$Value
transforms.ExtractTopicFromValueField.field.name=inner_field_name

License

This project is licensed under the Apache License, Version 2.0.

You can’t perform that action at this time.