@Antwnis Antwnis released this May 14, 2018 · 33 commits to master since this release

Assets 25

Release Notes

  • Upgrade to Kafka 1.1.0
  • Added SSL, subscription, partitioning, batching and key selection to Pulsar source and sink Elastic6 connector @caiooliveiraeti !
  • HTTP Basic Auth for Elasticsearch http client thanks @justinsoong !
  • Add polling timeout on the JMS source connector to avoid high CPU in the source connector poll thanks #373 @matthedude
  • Fixes on the elastic primary key separator thanks @caiooliveiraeti!
  • Fix on the MQTT class loader
  • Fix on the JMS class loader
  • Fix on JMS to close down connections cleanly #363 thanks @matthedude!
  • Fix on MQTT to correctly handle authentication
  • Moved MongoDB batch size to KCQL. connect.mongodb.batch.size is deprecated
  • Added connect.mapping.collection.to.json to treat maps, list, sets as json when inserting into Cassandra
  • Added support for Elastic Pipelines thanks @caiooliveiraeti!
  • Moved ReThinkDB batch size to KCQL connect.rethink.batch.size is deprecated
  • MQTT source allows full control of matching the topic INSERT INTO targetTopic SELECT * FROM mqttTopic ... WITHREGEX=`$THE_REGEX`
  • Upgrade Kudu Client to 0.7
  • Upgrade Azure documentDB client to 1.16.0
  • Upgrade Elastic5 to elastic4s 5.6.5
  • Upgrade Elastic6 to elastic4s 6.2.5
  • Upgrade Hazelcast client to 3.10
  • Upgrade InfluxDB client to 2.9
  • Upgrade MongoDB client to 3.6.3
  • Upgrade Redis client to 2.9
  • Kudu connector now accepts a comma separated list of master addresses
  • Added missing connect.elastic.retry.interval to elastic5 and elastic6
  • Added a default value set property to Cassandra to allow DEFAULT UNSET to be added on insert. Omitted columns from maps default to null. Alternatively, if set UNSET, pre-existing value will be preserved
  • Cassandra source batch size now in KCQL. connect.cassandra.batch.size is deprecated .
Jan 17, 2018
v0.4.0 for Kafka 0.11.x, release for independent libs

@andrewstevenson andrewstevenson released this Jan 16, 2018 · 135 commits to master since this release

Assets 24


Documentation available at http://lenses.stream

This release is for Kafka 0.11

  • Apache Pulsar - New Apache Pulsar source and sink Kafka connectors !
  • FTP - Add FTPS support via configuration ftp.protocol, either ftp (default) or ftps
  • MQTT source - Fix High CPU, thanks @masahirom
  • InfluxDB sink - Supports Dates and BigDecimals
  • Redis sink - Allow multiple primary keys
  • Kudu sink - Improve logging
  • JMS source - Support transacted queues #285 thanks @matthedude
  • Cassandra - Upgrade to Cassandra driver 3.3.0 and refactor tests
  • Cassandra sink - Added DELETE functionality when null null payload, thanks @sandonjacobs
  • Cassandra sink - Fix writing multiple topics to the same table in Cassandra #284
  • Cassandra source - Configurable timespan queries. You can now control the timespan the Connector will query for
  • Cassandra source - Allow setting start poll timestamp
  • Cassandra source - Allow setting initial query timestamp
  • kafka-connect-common - Support handling primary keys with doc strings, thanks @medvekoma

@stheppi stheppi released this Jan 4, 2018 · 147 commits to master since this release

Assets 3

We allow the records pulled from MQTT to be sent over to Kafka as JSON.
We also allow to pick a field(-s) from the incoming MQTT message and use them as the key for the Kafka message

Oct 25, 2017

@andrewstevenson andrewstevenson released this Sep 1, 2017 · 268 commits to master since this release

Assets 24

September-2017 release of (stream-reactor) for Kafka 0.11 and Confluent 3.3

  • Added MQTT Sink and wildcard support
  • JMS and MQTT connectors new KCQL support for WITHCONVERTERS and WITHTYPE
  • Added FLUSH MODE to Kudu. Thanks! @patsak
  • Upgrade CoAP to 2.0.0-M4.

Check 0.2.6 release notes below to find the complete CHANGELOG

@andrewstevenson andrewstevenson released this Sep 1, 2017 · 293 commits to master since this release

Assets 23

September-2017 release of (stream-reactor) for Kafka and Confluent 3.2.2


  • Added MQTT Sink
  • Upgrade to Confluent 3.2.2
  • Upgrade to KCQL 2x
  • Add CQL generator to Cassandra source
  • Add KCQL INCREMENTALMODE support to the Cassandra source, bulk mode and the timestamp column type is now take from KCQL
  • Support for setting key and truststore type on Cassandra connectors
  • Added token based paging support for Cassandra source
  • Added default bytes converter to JMS Source
  • Added default connection factory to JMS Source
  • Added support for SharedDurableConsumers to JMS Connectors
  • Upgraded JMS Connector to JMS 2.0
  • Moved to Elastic4s 2.4
  • Added Elastic5s with TCP, TCP+XPACK and HTTP client support
  • Upgrade Azure Documentdb to 1.11.0
  • Added optional progress counter to all connectors, it can be enabled with connect.progress.enabled which will periodically report log messages processed
  • Added authentication and TLS to ReThink Connectors
  • Added TLS support for ReThinkDB, add batch size option to source for draining the internal queues.
  • Upgrade Kudu Client to 1.4.0
  • Support for dates in Elastic Indexes and custom document types
  • Upgrade Connect CLI to 1.0.2 (Renamed to connect-cli)

Bug Fixes

  • Fixes for high CPU on CoAP source
  • Fixes for high CPU on Cassandra source
  • Fixed Avro double fields mapping to Kudu columns
  • Fixes on JMS properties converter, Invalid schema when extracting properties


  • Refactored Cassandra Tests to use only one embedded instance
  • Removed unused batch size and bucket size options from Kudu, they are taken from KCQL
  • Removed unused batch size option from DocumentDb
  • Rename Azure DocumentDb connect.documentdb.db to connect.documentdb.db
  • Rename Azure DocumentDb connect.documentdb.database.create to connect.documentdb.db.create
  • Rename Cassandra Source connect.cassandra.source.kcql to connect.cassandra.kcql
  • Rename Cassandra Source connect.cassandra.source.timestamp.type to connect.cassandra.timestamp.type
  • Rename Cassandra Source connect.cassandra.source.import.poll.interval to connect.cassandra.import.poll.interval
  • Rename Cassandra Source connect.cassandra.source.error.policy to connect.cassandra.error.policy
  • Rename Cassandra Source connect.cassandra.source.max.retries to connect.cassandra.max.retries
  • Rename Cassandra Sink connect.cassandra.source.retry.interval to connect.cassandra.retry.interval
  • Rename Cassandra Sink connect.cassandra.sink.kcql to connect.cassandra.kcql
  • Rename Cassandra Sink connect.cassandra.sink.error.policy to connect.cassandra.error.policy
  • Rename Cassandra Sink connect.cassandra.sink.max.retries to connect.cassandra.max.retries
  • Rename Cassandra Sink Sink connect.cassandra.sink.retry.interval to connect.cassandra.retry.interval
  • Rename Coap Source connect.coap.bind.port to connect.coap.port
  • Rename Coap Sink connect.coap.bind.port to connect.coap.port
  • Rename Coap Source connect.coap.bind.host to connect.coap.host
  • Rename Coap Sink connect.coap.bind.host to connect.coap.host
  • Rename MongoDb connect.mongo.database to connect.mongo.db
  • Rename MongoDb connect.mongo.sink.batch.size to connect.mongo.batch.size
  • Rename Druid connect.druid.sink.kcql to connect.druid.kcql
  • Rename Druid connect.druid.sink.conf.file to connect.druid.kcql
  • Rename Druid connect.druid.sink.write.timeout to connect.druid.write.timeout
  • Rename Elastic connect.elastic.sink.kcql to connect.elastic.kcql
  • Rename HBase connect.hbase.sink.column.family to connect.hbase.column.family
  • Rename HBase connect.hbase.sink.kcql to connect.hbase.kcql
  • Rename HBase connect.hbase.sink.error.policy to connect.hbase.error.policy
  • Rename HBase connect.hbase.sink.max.retries to connect.hbase.max.retries
  • Rename HBase connect.hbase.sink.retry.interval to connect.hbase.retry.interval
  • Rename Influx connect.influx.sink.kcql to connect.influx.kcql
  • Rename Influx connect.influx.connection.user to connect.influx.username
  • Rename Influx connect.influx.connection.password to connect.influx.password
  • Rename Influx connect.influx.connection.database to connect.influx.db
  • Rename Influx connect.influx.connection.url to connect.influx.url
  • Rename Kudu connect.kudu.sink.kcql to connect.kudu.kcql
  • Rename Kudu connect.kudu.sink.error.policy to connect.kudu.error.policy
  • Rename Kudu connect.kudu.sink.retry.interval to connect.kudu.retry.interval
  • Rename Kudu connect.kudu.sink.max.retries to connect.kudu.max.reties
  • Rename Kudu connect.kudu.sink.schema.registry.url to connect.kudu.schema.registry.url
  • Rename Redis connect.redis.connection.password to connect.redis.password
  • Rename Redis connect.redis.sink.kcql to connect.redis.kcql
  • Rename Redis connect.redis.connection.host to connect.redis.host
  • Rename Redis connect.redis.connection.port to connect.redis.port
  • Rename ReThink connect.rethink.source.host to connect.rethink.host
  • Rename ReThink connect.rethink.source.port to connect.rethink.port
  • Rename ReThink connect.rethink.source.db to connect.rethink.db
  • Rename ReThink connect.rethink.source.kcql to connect.rethink.kcql
  • Rename ReThink Sink connect.rethink.sink.host to connect.rethink.host
  • Rename ReThink Sink connect.rethink.sink.port to connect.rethink.port
  • Rename ReThink Sink connect.rethink.sink.db to connect.rethink.db
  • Rename ReThink Sink connect.rethink.sink.kcql to connect.rethink.kcql
  • Rename JMS connect.jms.user to connect.jms.username
  • Rename JMS connect.jms.source.converters to connect.jms.converters
  • Remove JMS connect.jms.converters and replace my kcql withConverters
  • Remove JMS connect.jms.queues and replace my kcql withType QUEUE
  • Remove JMS connect.jms.topics and replace my kcql withType TOPIC
  • Rename Mqtt connect.mqtt.source.kcql to connect.mqtt.kcql
  • Rename Mqtt connect.mqtt.user to connect.mqtt.username
  • Rename Mqtt connect.mqtt.hosts to connect.mqtt.connection.hosts
  • Remove Mqtt connect.mqtt.converters and replace my kcql withConverters
  • Remove Mqtt connect.mqtt.queues and replace my kcql withType=QUEUE
  • Remove Mqtt connect.mqtt.topics and replace my kcql withType=TOPIC
  • Rename Hazelcast connect.hazelcast.sink.kcql to connect.hazelcast.kcql
  • Rename Hazelcast connect.hazelcast.sink.group.name to connect.hazelcast.group.name
  • Rename Hazelcast connect.hazelcast.sink.group.password to connect.hazelcast.group.password
  • Rename Hazelcast connect.hazelcast.sink.cluster.members tp connect.hazelcast.cluster.members
  • Rename Hazelcast connect.hazelcast.sink.batch.size to connect.hazelcast.batch.size
  • Rename Hazelcast connect.hazelcast.sink.error.policy to connect.hazelcast.error.policy
  • Rename Hazelcast connect.hazelcast.sink.max.retries to connect.hazelcast.max.retries
  • Rename Hazelcast connect.hazelcast.sink.retry.interval to connect.hazelcast.retry.interval
  • Rename VoltDB connect.volt.sink.kcql to connect.volt.kcql
  • Rename VoltDB connect.volt.sink.connection.servers to connect.volt.servers
  • Rename VoltDB connect.volt.sink.connection.user to connect.volt.username
  • Rename VoltDB connect.volt.sink.connection.password to connect.volt.password
  • Rename VoltDB connect.volt.sink.error.policy to connect.volt.error.policy
  • Rename VoltDB connect.volt.sink.max.retries to connect.volt.max.retries
  • Rename VoltDB connect.volt.sink.retry.interval to connect.volt.retry.interval

@andrewstevenson andrewstevenson released this Jun 1, 2017 · 403 commits to master since this release

Assets 3

Release of Hbase sink connector only for addressing ByteBuffer conversions for #204