Skip to content

Releases: ClickHouse/clickhouse-kafka-connect

v1.5.0 (2026-08-05)

Choose a tag to compare

@github-actions github-actions released this 05 Aug 17:53
483f7c4

Improvements

  • RowBinary inserts with client V2 now stream data directly to the network output stream via the client's
    DataStreamWriter API. Increase in performance is ~ 30% for 100k dataset. (#796)

  • Support Avro nullable multi-type unions (e.g. [null, string, int]) mapping to a ClickHouse Variant.
    Confluent's Avro converter turns the non-null branches into a Connect union struct
    (io.confluent.connect.avro.Union), with the null branch making the field optional; the connector
    resolves this to Variant(String, Int32). (#799)

Dependencies

  • [build] Updated com.diffplug.spotless from 8.6.0 to 8.9.0.
  • [build] Updated gradle-wrapper from 9.5.1 to 9.6.1
  • [tests] Updated com.squareup.okhttp3:okhttp from 5.3.2 to 5.4.0.

Bug Fixes

  • Fixed issue with Table Schema cache that can remember old schema instead of new one. Now if old
    schema is read proper number of columns is calculated and cache will be properly updated after some time.
    (#813)

v1.4.0 (2026-07-15)

Choose a tag to compare

@github-actions github-actions released this 15 Jul 20:53
eee0ec3

Security

  • Upgraded com.fasterxml.jackson.core dependencies to version 2.21.5 to address multiple CVE's in previous releases.

New Features

  • Internal buffering now supports exactlyOnce=true via strict-chunking mode. When both bufferCount > 0 and
    exactlyOnce=true, records are bucketed per (topic, partition) and flushed only in fixed bufferCount-sized
    chunks. Tail records below the threshold remain buffered until subsequent put() calls grow the bucket past
    bufferCount. This keeps (minOffset, maxOffset) reproducible across retries, allowing ClickHouse
    insert_deduplication_token reuse and StateProvider range comparison to work correctly. Requires bufferFlushTime=0
    and ignorePartitionsWhenBatching=false — the start-up validator throws ConnectException otherwise.

Improvements

  • Cast/conversion failures in ClickHouseWriter now surface the failing column and its types.
    Previously a type mismatch (e.g. a java.util.Date reaching a UInt64 column) threw a bare
    ClassCastException that named only Java types, and the useful context — column name, target
    ClickHouse type, source Kafka type — was written to the container logs only. The connector now
    rethrows these as a non-retryable DataException carrying that context, so it also reaches the Kafka
    Connect REST status, DLQ record headers, and JMX task-error-metrics. Record values are never
    included in the message. (#729)

Internal Changes

  • Refactored ClickHouseSinkTask to delegate to a DeliveryStrategy per delivery semantic (DirectDeliveryStrategy,
    AtLeastOnceBufferStrategy, ExactlyOnceBufferStrategy), with a shared ChunkFlusher for the insert +
    offset-tracking path. The buffering and non-buffering flows now live in separate, cohesive units instead of
    interleaved branches on the task. Behavior is unchanged.

Bug Fixes

  • Fixed NullPointerException when writing a null value into a Nullable(JSON) column via the binary
    insert path with input_format_binary_read_json_as_string=1. The JSON case in ClickHouseWriter cast the
    field straight to String without checking for null first, unlike every other nullable-aware type in the
    same switch. (#562)

Dependencies

  • Bumped org.json:json from 20250517 to 20260522.

v1.3.10 (2026-06-24)

Choose a tag to compare

@github-actions github-actions released this 25 Jun 05:38

New Features

  • Added clusterName configuration option. When set, the connector includes an ON CLUSTER '<name>'
    clause in the distributed DDL it runs (e.g. the ALTER TABLE ... ADD COLUMN statements issued by
    auto.evolve), so schema evolution propagates across all replicas in a clustered deployment. Empty by
    default. (#716, #738)

Bug Fixes

  • Inserts now time out instead of blocking forever when ClickHouse becomes unresponsive (network partition,
    server hang, long GC pause). Previously a stuck insert could hold the task past
    consumer.override.max.poll.interval.ms, causing Kafka to rebalance the partitions mid-insert. The new
    clickhouseClientInsertTimeoutMs setting (default 240000) bounds the wait; on timeout a retriable
    exception is thrown so the Connect framework retries the batch. Set this below
    consumer.override.max.poll.interval.ms. (#756)

Improvements

  • Reduced log noise: messages about unhandled complex sub-types encountered while building table mappings
    are now logged at DEBUG instead of WARN, since they are expected and not errors.
    (#767)

Dependencies

  • Bumped org.projectlombok:lombok from 1.18.38 to 1.18.46.
  • Bumped org.apache.httpcomponents.client5:httpclient5 (test scope) from 5.5 to 5.6.1.

Build

  • Upgraded Gradle from 7.4.2 to 9.5.1 and migrated dependency management to a Gradle version catalog
    (gradle/libs.versions.toml) and the jvm-test-suite plugin.

v1.3.9 (2026-05-21)

Choose a tag to compare

@github-actions github-actions released this 22 May 17:45
0dd4cc2

Bug Fixes

  • To gracefully handle live schema drift between the connector and ClickHouse, refresh the table mappings when connector encounters Code: 131. DB::Exception: Too large string size and log column-level schema updates. Now, the connector refreshes its table mappings automatically when ClickHouse returns either Code: 33 or Code: 131 and retries the insert. (#751)

v1.3.8 (2026-05-08)

Choose a tag to compare

@github-actions github-actions released this 12 May 00:40
36f10db

New Features

  • Added auto.evolve configuration option for automatic table schema evolution. When enabled, the connector
    detects new fields in incoming records and issues ALTER TABLE ... ADD COLUMN IF NOT EXISTS against ClickHouse.
    Disabled by default. (#277)

Bug Fixes

  • Fixed RowBinary serialization for Map columns with Nullable value types. The nullable marker byte was missing
    when writing map values, causing CANNOT_READ_ALL_DATA errors for Map(K, Nullable(V)) columns. (#687)
  • Fixed com.clickhouse.kafka.connect.transforms.KeyToValue transformation to handle different schemas. Before
    this fix, the value schema was cached and not updated, making the transformation incompatible with evolving
    schemas. (#718)

Dependencies

  • Updated clickhouse-java version from 0.9.4 to 0.9.5

v1.3.7 (2026-03-25)

Choose a tag to compare

@github-actions github-actions released this 26 Mar 17:00
742bc9a

Security

Improvements

  • Gson replaced with Jackson for performance and better maintainability (#676).

v1.3.6

Choose a tag to compare

@github-actions github-actions released this 18 Mar 20:55
c1ab031

New Features

  • Added internal record buffering support via bufferCount and bufferFlushTime configuration options.
    When enabled, records from multiple poll() calls are accumulated and flushed as a single large batch,
    reducing the number of insert operations to ClickHouse. Buffering is disabled by default (bufferCount=0) for full backward compatibility. (#658)

Improvements

  • Report inserted offsets in preCommit() method. Previously connector was returning same map that is passed to
    the method. This may lead to missed offsets in a situation of partition rebalance. Feature is turned off
    by default and reportInsertedOffsets property should be set to true to enable. (#669)

Bug Fixes

  • Fixed invalid concurrency handling in ClickHouseWriter.updateMapping. Previously flag was set not in atomic way. (#678)
  • Fixes handling server error "Code 33" after schema is updated. Previously logic did not wait for target table to be updated.
    After the fix logic will use describe of a single table when detects that updateMapping is running. (#680)

v1.3.5

Choose a tag to compare

@github-actions github-actions released this 18 Dec 19:08
fd18750

Important

Upgrade to this version if working with ClickHouse 25.10+. Older version has issues with compression because uses older Java Client. Which fixes this issue in the 0.9.4 release ( https://github.com/ClickHouse/clickhouse-java/releases ).

Improvements

  • Added topic metrics to JMX and extended task metrics. Topic metrics have partition granularity - each partition has its own metrics (#612)
  • Improved failure handling when messages should be sent to DLQ. Previous implementation was sending whole batch to DLQ
    when one message failed. Now it sends only failed group of messages to DLQ. If failed because of schema validation then
    logs error message contains field name and schema type. (#590)
  • Added support for writing boolean values to number columns. This is allowed because boolean fits into any number type. (#633)

Bug Fixes

  • Fixed error message about unhandled complex type in array (#608)
  • Fixed logging in schema validation logic. Message "Got non-root column, but its parent was not found to be updated"
    was logged as error but should be a warning. (#645)
  • Fixed writing Avro timestamp value. (#599)
  • Fixed negative timestamp in logs. (#614)

Dependencies

  • Updated clickhouse-java version to 0.9.4 (#629)

v1.3.4

Choose a tag to compare

@github-actions github-actions released this 09 Oct 03:02
4d8b433
  • Added column named that causes error (#607)
  • Updated clickhouse-java version (#596)
  • Fixed Client Name reported for query_log (#542)
  • Fixed writing Avro union of string and bytes to String column (#572)

v1.3.3

Choose a tag to compare

@github-actions github-actions released this 30 Aug 15:12
62450ce
  • Fixed writing JSON values to ClickHouse. Previously all JSON Object were written as objects where field value were wrapped with { "object": <field value> }. Now objects stored with original structure. (#574)
  • Added support of SimpleAggregateFunction column type. (#571)