Releases: ClickHouse/clickhouse-kafka-connect
Releases · ClickHouse/clickhouse-kafka-connect
Release list
v1.5.0 (2026-08-05)
Improvements
-
RowBinary inserts with client V2 now stream data directly to the network output stream via the client's
DataStreamWriterAPI. Increase in performance is ~ 30% for 100k dataset. (#796) -
Support Avro nullable multi-type unions (e.g.
[null, string, int]) mapping to a ClickHouseVariant.
Confluent's Avro converter turns the non-null branches into a Connect union struct
(io.confluent.connect.avro.Union), with thenullbranch making the field optional; the connector
resolves this toVariant(String, Int32). (#799)
Dependencies
- [build] Updated
com.diffplug.spotlessfrom 8.6.0 to 8.9.0. - [build] Updated
gradle-wrapper from9.5.1 to 9.6.1 - [tests] Updated
com.squareup.okhttp3:okhttpfrom 5.3.2 to 5.4.0.
Bug Fixes
- Fixed issue with Table Schema cache that can remember old schema instead of new one. Now if old
schema is read proper number of columns is calculated and cache will be properly updated after some time.
(#813)
v1.4.0 (2026-07-15)
Security
- Upgraded
com.fasterxml.jackson.coredependencies to version2.21.5to address multiple CVE's in previous releases.
New Features
- Internal buffering now supports
exactlyOnce=truevia strict-chunking mode. When bothbufferCount > 0and
exactlyOnce=true, records are bucketed per(topic, partition)and flushed only in fixedbufferCount-sized
chunks. Tail records below the threshold remain buffered until subsequentput()calls grow the bucket past
bufferCount. This keeps(minOffset, maxOffset)reproducible across retries, allowing ClickHouse
insert_deduplication_tokenreuse and StateProvider range comparison to work correctly. RequiresbufferFlushTime=0
andignorePartitionsWhenBatching=false— the start-up validator throwsConnectExceptionotherwise.
Improvements
- Cast/conversion failures in
ClickHouseWriternow surface the failing column and its types.
Previously a type mismatch (e.g. ajava.util.Datereaching aUInt64column) threw a bare
ClassCastExceptionthat named only Java types, and the useful context — column name, target
ClickHouse type, source Kafka type — was written to the container logs only. The connector now
rethrows these as a non-retryableDataExceptioncarrying that context, so it also reaches the Kafka
Connect REST status, DLQ record headers, and JMXtask-error-metrics. Record values are never
included in the message. (#729)
Internal Changes
- Refactored
ClickHouseSinkTaskto delegate to aDeliveryStrategyper delivery semantic (DirectDeliveryStrategy,
AtLeastOnceBufferStrategy,ExactlyOnceBufferStrategy), with a sharedChunkFlusherfor the insert +
offset-tracking path. The buffering and non-buffering flows now live in separate, cohesive units instead of
interleaved branches on the task. Behavior is unchanged.
Bug Fixes
- Fixed
NullPointerExceptionwhen writing anullvalue into aNullable(JSON)column via the binary
insert path withinput_format_binary_read_json_as_string=1. The JSON case inClickHouseWritercast the
field straight toStringwithout checking fornullfirst, unlike every other nullable-aware type in the
same switch. (#562)
Dependencies
- Bumped
org.json:jsonfrom20250517to20260522.
v1.3.10 (2026-06-24)
New Features
- Added
clusterNameconfiguration option. When set, the connector includes anON CLUSTER '<name>'
clause in the distributed DDL it runs (e.g. theALTER TABLE ... ADD COLUMNstatements issued by
auto.evolve), so schema evolution propagates across all replicas in a clustered deployment. Empty by
default. (#716, #738)
Bug Fixes
- Inserts now time out instead of blocking forever when ClickHouse becomes unresponsive (network partition,
server hang, long GC pause). Previously a stuck insert could hold the task past
consumer.override.max.poll.interval.ms, causing Kafka to rebalance the partitions mid-insert. The new
clickhouseClientInsertTimeoutMssetting (default240000) bounds the wait; on timeout a retriable
exception is thrown so the Connect framework retries the batch. Set this below
consumer.override.max.poll.interval.ms. (#756)
Improvements
- Reduced log noise: messages about unhandled complex sub-types encountered while building table mappings
are now logged atDEBUGinstead ofWARN, since they are expected and not errors.
(#767)
Dependencies
- Bumped
org.projectlombok:lombokfrom1.18.38to1.18.46. - Bumped
org.apache.httpcomponents.client5:httpclient5(test scope) from5.5to5.6.1.
Build
- Upgraded Gradle from
7.4.2to9.5.1and migrated dependency management to a Gradle version catalog
(gradle/libs.versions.toml) and thejvm-test-suiteplugin.
v1.3.9 (2026-05-21)
Bug Fixes
- To gracefully handle live schema drift between the connector and ClickHouse, refresh the table mappings when connector encounters
Code: 131. DB::Exception: Too large string sizeand log column-level schema updates. Now, the connector refreshes its table mappings automatically when ClickHouse returns eitherCode: 33orCode: 131and retries the insert. (#751)
v1.3.8 (2026-05-08)
New Features
- Added
auto.evolveconfiguration option for automatic table schema evolution. When enabled, the connector
detects new fields in incoming records and issuesALTER TABLE ... ADD COLUMN IF NOT EXISTSagainst ClickHouse.
Disabled by default. (#277)
Bug Fixes
- Fixed RowBinary serialization for Map columns with Nullable value types. The nullable marker byte was missing
when writing map values, causingCANNOT_READ_ALL_DATAerrors forMap(K, Nullable(V))columns. (#687) - Fixed
com.clickhouse.kafka.connect.transforms.KeyToValuetransformation to handle different schemas. Before
this fix, the value schema was cached and not updated, making the transformation incompatible with evolving
schemas. (#718)
Dependencies
- Updated clickhouse-java version from
0.9.4to0.9.5
v1.3.7 (2026-03-25)
Security
- Upgraded
com.fasterxml.jackson.coredependencies to version with fix for GHSA-72hv-8253-57qq (#690).
Improvements
Gsonreplaced withJacksonfor performance and better maintainability (#676).
v1.3.6
New Features
- Added internal record buffering support via
bufferCountandbufferFlushTimeconfiguration options.
When enabled, records from multiplepoll()calls are accumulated and flushed as a single large batch,
reducing the number of insert operations to ClickHouse. Buffering is disabled by default (bufferCount=0) for full backward compatibility. (#658)
Improvements
- Report inserted offsets in
preCommit()method. Previously connector was returning same map that is passed to
the method. This may lead to missed offsets in a situation of partition rebalance. Feature is turned off
by default andreportInsertedOffsetsproperty should be set totrueto enable. (#669)
Bug Fixes
- Fixed invalid concurrency handling in
ClickHouseWriter.updateMapping. Previously flag was set not in atomic way. (#678) - Fixes handling server error "Code 33" after schema is updated. Previously logic did not wait for target table to be updated.
After the fix logic will use describe of a single table when detects thatupdateMappingis running. (#680)
v1.3.5
Important
Upgrade to this version if working with ClickHouse 25.10+. Older version has issues with compression because uses older Java Client. Which fixes this issue in the 0.9.4 release ( https://github.com/ClickHouse/clickhouse-java/releases ).
Improvements
- Added topic metrics to JMX and extended task metrics. Topic metrics have partition granularity - each partition has its own metrics (#612)
- Improved failure handling when messages should be sent to DLQ. Previous implementation was sending whole batch to DLQ
when one message failed. Now it sends only failed group of messages to DLQ. If failed because of schema validation then
logs error message contains field name and schema type. (#590) - Added support for writing boolean values to number columns. This is allowed because boolean fits into any number type. (#633)
Bug Fixes
- Fixed error message about unhandled complex type in array (#608)
- Fixed logging in schema validation logic. Message "Got non-root column, but its parent was not found to be updated"
was logged as error but should be a warning. (#645) - Fixed writing Avro timestamp value. (#599)
- Fixed negative timestamp in logs. (#614)
Dependencies
- Updated clickhouse-java version to
0.9.4(#629)