Releases: firebolt-db/firebolt-kafka-connector
Releases · firebolt-db/firebolt-kafka-connector
Release list
v0.4.4
What's changed
Distribution
- The Confluent Hub archive (
firebolt-db-firebolt-kafka-connect-0.4.4.zip) is now published alongside the JARs on this release page. Use it withconfluent-hub install, with Confluent Cloud Custom Connectors, or with self-hosted Kafka Connect (drop the unzipped contents into yourplugin.path).
Security & hardening
- Internal hardening of integration-test code and CI workflows. No behavioural change to the connector itself.
Compatibility
- Tested against Kafka Connect 3.9.1 and 4.0.
Assets
firebolt-db-firebolt-kafka-connect-0.4.4.zip— Confluent Hub archive (preferred forconfluent-hub installand Custom Connectors)firebolt-kafka-connector-0.4.4.jar— uber-jar for self-hosted Kafka Connect (drop intoplugin.path)firebolt-kafka-connector-0.4.4-sources.jar— source archive
Full Changelog: v0.4.3...v0.4.4
v0.4.3
What's Changed
- build(NoTicket): bump gradle/actions from 5 to 6 by @dependabot[bot] in #269
- fix: patch nimbus-jose-jwt (CVE-2025-53864) and jetty-http (CVE-2024-6763, CVE-2025-11143) by @asimrout-eng in #270
Full Changelog: v0.4.2...v0.4.3
v0.4.2
What's Changed
- fix: patch jackson-core (GHSA-72hv-8253-57qq) and aircompressor (CVE-2025-67721) by @asimrout-eng in #268
New Contributors
- @asimrout-eng made their first contribution in #268
Full Changelog: v0.4.1...v0.4.2
v0.4.1
What's Changed
- ci: fix connector creation in load tests (
FIR-52465) by @goprean in #238 - feat: add support for avro ingestion of integer (
FIR-52468) by @goprean in #239 - feat: add support for avro ingestion of bigint (
FIR-52472) by @goprean in #240 - feat: added support for numeric data type in avro (
FIR-52473) by @goprean in #241 - feat: add support for avro ingestion of real (
FIR-52474) (#240) by @goprean in #242 - feat: add support for avro ingestion of double precision (
FIR-52475) by @goprean in #243 - feat: add support for avro ingestion of boolean (
FIR-52476) by @goprean in #244 - feat: add support for avro ingestion of date (
FIR-52477) by @goprean in #245 - feat: add support for avro ingestion of timestamp (
FIR-52478) by @goprean in #249 - feat: added support for timestamptz with avro format (
FIR-52479) by @goprean in #250 - feat: added support for bytea data type in avro (
FIR-52480) by @goprean in #251 - feat: added support for text data type with avro format (
FIR-52481) by @goprean in #252 - Revert "ci: fix connector creation in load tests (
FIR-52465) (#238)" by @goprean in #256 - test: add a test with all data types with avro format (
FIR-52512) by @goprean in #254 - feat: added support for json data type with avro (
FIR-52482) by @goprean in #253 - test: added ability to run load tests using avro format (
FIR-52510) by @goprean in #257
Full Changelog: v0.4.0...v0.4.1
v0.4.0
What's Changed
- ci: run load test continuously (
FIR-51592) by @goprean in #232 - feat: added support for json column data type (
FIR-51880) by @goprean in #230 - ci: capture some release notes summary in addition to version number (NoTicket) by @goprean in #234
- ci: pass in the firebolt image tag as a github variable (NoTicket) by @goprean in #231
- ci: parameterize the scenario and message count for continuous load testing (NoTicket) by @goprean in #236
- test: run parquet binary ingestion for core too (
FIR-51020) by @goprean in #235 - refactor: use the manifest file from confluent (NoTicket) by @goprean in #237
Full Changelog: v0.3.1...v0.4.0
v0.3.1
What's Changed
- ci: the version of the connector was missing from the slack message
FIR-51418by @goprean in #189 - refactor: change the class names to binary to reflect the ingestion type
FIR-51325by @goprean in #181 - feat: add schema support for parquet ingestion for integer
FIR-51325by @goprean in #176 - feat: add schema support for parquet ingestion for bigint
FIR-51325by @goprean in #177 - feat: add schema support for timestamp binary ingestion
FIR-51325by @goprean in #190 - feat: add schema support for timestamptz binary ingestion
FIR-51325by @goprean in #191 - feat: add schema support for real binary ingestion
FIR-51325by @goprean in #192 - feat: add schema support for double binary ingestion
FIR-51325by @goprean in #193 - feat: add schema support for decimal binary ingestion
FIR-51325by @goprean in #194 - feat: add schema support for date binary ingestion
FIR-51325by @goprean in #195 - feat: add schema support for text binary ingestion
FIR-51325by @goprean in #196 - feat: add schema support for bytea binary ingestion
FIR-51325by @goprean in #197 - feat: add schema support for boolean binary ingestion
FIR-51325by @goprean in #198 - refactor(NoTicket): moved the prepared statement classes under ingestion/sql by @goprean in #199
- build(NoTicket): bump org.sonarqube from 7.2.1.6560 to 7.2.2.6593 by @dependabot[bot] in #201
- build(NoTicket): bump testcontainersVersion from 1.21.3 to 1.21.4 by @dependabot[bot] in #202
- build(NoTicket): bump log4jVersion from 2.25.2 to 2.25.3 by @dependabot[bot] in #203
- test: run parquet tests on cloud
FIR-51519by @goprean in #200 - test: add support for schemaless load test run
FIR-51592by @goprean in #206 - test: added load tests for table with 1000 columns
FIR-51588by @goprean in #204 - test: randomize the payload of the kafka messages
FIR-51592by @goprean in #208 - test: add one workflow to run multiple load tests combinations
FIR-51592by @goprean in #205 - test: fix merge issue
FIR-51592by @goprean in #209 - test: also need to set the partition min bytes
FIR-51592by @goprean in #210 - test: increase timeout for load-test to 5hrs
FIR-51592by @goprean in #212 - ci: do not truncate log messages
FIR-51592by @goprean in #213 - test: combination of sql and 1000 columns causes out of memory exceptions on the connector by @goprean in #211
- test: download the connector logs and upload them as artifacts to the job
FIR-51592by @goprean in #214 - build(NoTicket): bump io.firebolt:firebolt-jdbc from 3.9.0 to 3.9.1 by @dependabot[bot] in #216
- test: add scenario based testing
FIR-51592by @goprean in #207 - test: missed the min fetch byte for scenario based
FIR-51592by @goprean in #218 - test: set the max bytes to the received input
FIR-51592by @goprean in #219 - build(NoTicket): bump io.firebolt:firebolt-jdbc from 3.9.1 to 3.10.0 by @dependabot[bot] in #220
- build(NoTicket): bump org.apache.commons:commons-csv from 1.11.0 to 1.14.1 by @dependabot[bot] in #222
- build(NoTicket): bump parquetVersion from 1.16.0 to 1.17.0 by @dependabot[bot] in #223
- test: run scenario load tests with post processing
FIR-51592by @goprean in #225 - build(NoTicket): bump jacksonVersion from 2.19.2 to 2.21.0 by @goprean in #226
- test: added scenario2 load testing
FIR-51592by @goprean in #227
Full Changelog: v0.3...v0.3.1
v0.3
What's Changed
- feat: add exactly once feature flag
FIR-49600by @goprean in #78 - build(NoTicket): bump log4jVersion from 2.25.1 to 2.25.2 by @dependabot[bot] in #80
- build(NoTicket): bump org.apache.commons:commons-lang3 from 3.18.0 to 3.19.0 by @dependabot[bot] in #84
- feat(tests): allow json messages without registry
FIR-49734by @goprean in #85 - feat(tests): add more integration tests for each supported data type by @goprean in #86
- feat: allow consumption of kafka message without schema registry
FIR-49734by @goprean in #87 - feat: fix the float serializartion/deserialization for large/small floats
FIR-49832by @goprean in #89 - refactor: moved the tests and the convertes for schema into a schema package
FIR-49860by @goprean in #90 - refactor: moved converters and tests to schema package
FIR-49860by @goprean in #91 - build(NoTicket): bump okhttpVersion from 5.1.0 to 5.2.1 by @dependabot[bot] in #93
- refactor: the way its decided which column converter to use by @goprean in #96
- refactor: added the kafka message column value interface
FIR-49860by @goprean in #97 - feat: added the ability to use post processing script -
FIR-49943by @goprean in #98 - feat: allow the post-script execution at the end of a batch by @goprean in #99
- build(NoTicket): bump org.apache.avro:avro from 1.12.0 to 1.12.1 by @dependabot[bot] in #102
- build(NoTicket): bump org.sonarqube from 6.3.1.5724 to 7.0.0.6105 by @dependabot[bot] in #103
- build(NoTicket): bump org.apache.commons:commons-text from 1.12.0 to 1.14.0 by @dependabot[bot] in #100
- refactor: part5 refactor schema and schemaless base test classes (
FIR-49860) by @goprean in #105 - feat:
FIR-49576Implement MetadataService by @bogdantruta-firebolt in #79 - test: Added more verifications to MetadataService unit tests
FIR-49576by @goprean in #107 - Fir 49574 work for append only exactly once by @bogdantruta-firebolt in #106
- build(NoTicket): bump org.sonarqube from 7.0.0.6105 to 7.0.1.6134 by @dependabot[bot] in #111
- bug: allow saving a text column as proper json
FIR-50187by @goprean in #110 - build(NoTicket): bump io.firebolt:firebolt-jdbc from 3.7.1 to 3.7.2 by @dependabot[bot] in #114
- build(NoTicket): bump okhttpVersion from 5.2.1 to 5.3.0 by @dependabot[bot] in #116
- bug: fix issue seen by customer when casting to int32
FIR-50549by @goprean in #120 - feat: use compression of the payload by default by @goprean in #121
- bug: rollback the transaction if commit fails
FIR-54476by @goprean in #117 - feat: optimize the null columns for all records in a batch
FIR-50460by @goprean in #118 - Fir 50622 add property for optimizing inserts by @bogdantruta-firebolt in #124
- feat:
FIR-50334allow setting of a post processing script file by @bogdantruta-firebolt in #123 - ci: automate the release process
FIR-50782by @goprean in #125 - refactor: prepare for adding a new ingestion service
FIR-50864by @goprean in #127 - feat: add a new ingestion type config definition
FIR-50864by @goprean in #128 - build(NoTicket): bump org.sonarqube from 7.0.1.6134 to 7.1.0.6387 by @dependabot[bot] in #135
- build(NoTicket): bump org.apache.commons:commons-lang3 from 3.19.0 to 3.20.0 by @dependabot[bot] in #136
- build(NoTicket): bump okhttpVersion from 5.3.0 to 5.3.2 by @dependabot[bot] in #137
- feat: transform kafka messages to parquet file format
FIR-50865by @goprean in #126 - refactor: use the decorator pattern for ingestion service with post-processing
FIR-50864by @goprean in #130 - feat: add the binary ingestion service
FIR-50865by @goprean in #129 - refactor: prepare for running integration tests against sql and binary ingestion
FIR-50984by @goprean in #131 - feat: decide which ingestion service to use based on the ingestion.type definition
FIR-50865by @goprean in #132 - feat: add consuming integer messages using parquet
FIR-50866by @goprean in #134 - feat: support bigint ingestion with parquet
FIR-50868by @goprean in #138 - feat: allow timestamp conversion to parquet
FIR-50872by @goprean in #139 - feat: support conversion of timestamptz to parquet format
FIR-50873by @goprean in #141 - feat: support ingestion of real data type using parquer format
FIR-50874by @goprean in #142 - feat: allow ingestion of decimal data type using parquet
FIR-50875by @goprean in #143 - feat: allow ingestion of double precision data using parquet
FIR-50877by @goprean in #144 - feat: allow ingestion of date data type using parquet
FIR-50876by @goprean in #145 - feat: allow text data type ingestion using parquet
FIR-50867by @goprean in #146 - revert:
FIR-51068revert optimize imports config by @bogdantruta-firebolt in #149 - feat: allow ingestion of boolean data type using parquet
FIR-50871by @goprean in #148 - feat:
FIR-51063Change temporary http client to jdbc feature by @bogdantruta-firebolt in #147 - feat: allow ingestion of bytea using parquet format
FIR-50878by @goprean in #150 - refactor: extract the message generation and producing out of load test runner
FIR-51114by @goprean in #152 - refactor: take out the verifier outside of the runner
FIR-51114by @goprean in #153 - bug: allow parquet ingestion for tables with dashes in their name by @goprean in #154
- refactor: use the message sequenceId in the TestFactory
FIR-51114by @goprean in #156 - refactor: make sure we perform cleanup when laod tests fail
FIR-51114by @goprean in #158 - ci: add the workflow to add a new connector plugin
FIR-51190by @goprean in #157 - refactor: prepare load test to be run using binary ingestion
FIR-51114by @goprean in #155 - docs: add some demo scripts
FIR-51117by @goprean in #151 - bug: allow column names in table schema that are not valid names in avro
FIR-51145by @goprean in #159 - bug: allow ingestion when column name and json attributes match by case insensitive equality
FIR-51145by @goprean in #160 - build(NoTicket): bump org.sonarqube from 7.1.0.6387 to 7.2.0.6526 by @dependabot[bot] in #166
- buil...
v0.2
What's Changed
- build(NoTicket): bump mockitoVersion from 5.18.0 to 5.19.0 by @dependabot[bot] in #39
- (perf) FIR-48535 performance load testing against the connector by @goprean in #41
- (perf) FIR-48535 Remove the runtime hook by @goprean in #42
- (ci) NoTicket - fix the firebolt core directory permission by @goprean in #44
- (deps) FIR-48534 add the commons-lang3 dependency by @goprean in #43
- build(NoTicket): bump io.firebolt:firebolt-jdbc from 3.6.7 to 3.7.0 by @dependabot[bot] in #46
- (feat) FIR-48534 batch records in smaller chunks if it exceeds the max org limit in firebolt by @goprean in #45
- FIR-48534 use the exception type to detect when the payload is too large by @goprean in #47
- (bug) FIR-48703 removing topic from topics definition attribute still process that topic by @goprean in #48
- (feat) FIR-48781 allow certain strings to be converted to values in boolean column data type by @goprean in #49
- Fir 48782 boolean failed conversion by @goprean in #50
- (feat) FIR-48782 - records with int columns that cannot be converted move to the deadletter queue by @goprean in #51
- (feat) FIR-48782 allow smaller integer types to be saved in bigint column by @goprean in #52
- (feat) FIR-48782 - records with real data columns that cannot be converted move to the deadletter queue by @goprean in #53
- (feat) FIR-48782 - records with double precision columns that cannot be converted move to the deadletter queue by @goprean in #54
- build(NoTicket): bump org.sonarqube from 6.2.0.5505 to 6.3.1.5724 by @dependabot[bot] in #58
- (feat) FIR-48782 - records with numeric columns that cannot be converted are moved to the deadletter queue by @goprean in #55
- (feat) FIR-48782 - records with date columns that cannot be converted move to the deadletter queue by @goprean in #59
- (feat) FIR-48782 records with timestamp columns that cannot be converted will be move to the deadletter queue by @goprean in #60
- (feat) FIR-48782 records with timestamptz columns that cannot be converted will be move to the deadletter queue by @goprean in #61
- (feat) FIR-48782 records with string columns that cannot be converted will be moved to the deadletter queue by @goprean in #62
- (feat) FIR-48782 records with bytea columns that cannot be converted will be moved to the deadletter queue by @goprean in #63
- build(NoTicket): bump org.projectlombok:lombok from 1.18.38 to 1.18.40 by @dependabot[bot] in #65
- (feat) FIR-48916 Do not allow connecting without account, engine or database to Firebolt Cloud by @goprean in #64
- (ci) FIR-49021 add a git workflow that lists all the custom plugins with firebolt in the name by @goprean in #68
- feat: FIR-48541 add support for dead letter queue by @bogdantruta-firebolt in #66
- refactor: FIR-48541 removing unused tests and reducing FireboltRecord list by @bogdantruta-firebolt in #69
- build(NoTicket): bump org.projectlombok:lombok from 1.18.40 to 1.18.42 by @dependabot[bot] in #72
- build(NoTicket): bump mockitoVersion from 5.19.0 to 5.20.0 by @dependabot[bot] in #73
- build(NoTicket): bump com.google.guava:guava from 33.4.8-jre to 33.5.0-jre by @dependabot[bot] in #71
- Fix the load tests running on latest connector by @goprean in #74
- (feat) FIR-48662 group backend exceptions into recoverable and non-recoverable exceptions by @goprean in #75
- feat: release version 0.2 (FIR-49394) by @goprean in #76
- chore: Naming convention should be firebolt-db-firebolt-kafka-connector-.zip by @goprean in #77
New Contributors
- @bogdantruta-firebolt made their first contribution in #66
Full Changelog: v0.1...v0.2