Lenses for Apache Kafka
Lenses offers SQL (for data browsing and Kafka Streams), Kafka Connect connector management, cluster monitoring and more.
You can find more on lenses.io
A collection of components to build a real time ingestion pipeline.
- Kafka 2.8 -> 3.3 (Confluent 2.8 -> 7.3) - Upcoming Stream Reactor 4.1.0
- Kafka 3.1 (Confluent 7.1) - Stream Reactor 4.0.0 (Kafka 3.1 Build)
- Kafka 2.8 (Confluent 6.2) - Stream Reactor 4.0.0 (Kafka 2.8 Build)
- Kafka 2.5 (Confluent 5.5) - Stream reactor 2.0.0+
- Kafka 2.0 -> 2.4 (Confluent 5.4) - Stream reactor 1.2.7
The following connectors have been deprecated and are no longer included in any release from 3.0.
The following connectors have been deprecated and are no longer included in any release from 4.0.
- Hive 1.1
Please take a moment and read the documentation and make sure the software prerequisites are met!!
|AWS S3||Sink||Copy data from Kafka to AWS S3.||Docs|
|AzureDocumentDb||Sink||Copy data from Kafka and Azure Document Db.||Docs|
|Cassandra||Source||Copy data from Cassandra to Kafka.||Docs|
|*Cassandra||Sink||Certified DSE Cassandra, copy data from Kafka to Cassandra.||Docs|
|Elastic 6||Sink||Copy data from Kafka to Elastic Search 6.x w. tcp or http||Docs|
|FTP/HTTP||Source||Copy data from FTP/HTTP to Kafka.||Docs|
|Hazelcast||Sink||Copy data from Kafka to Hazelcast.||Docs|
|HBase||Sink||Copy data from Kafka to HBase.||Docs|
|Hive||Source||Copy data from Hive/HDFS to Kafka.||Docs|
|Hive||Sink||Copy data from Kafka to Hive/HDFS||Docs|
|InfluxDb||Sink||Copy data from Kafka to InfluxDb.||Docs|
|Kudu||Sink||Copy data from Kafka to Kudu.||Docs|
|JMS||Source||Copy data from JMS topics/queues to Kafka.||Docs|
|JMS||Sink||Copy data from Kafka to JMS.||Docs|
|MongoDB||Sink||Copy data from Kafka to MongoDB.||Docs|
|MQTT||Source||Copy data from MQTT to Kafka.||Docs|
|MQTT||Sink||Copy data from Kafka to MQTT.||Docs|
|Pulsar||Source||Copy data from Pulsar to Kafka.||Docs|
|Pulsar||Sink||Copy data from Kafka to Pulsar.||Docs|
|Redis||Sink||Copy data from Kafka to Redis.||Docs|
Please see the Stream Reactor Release Notes at Lenses Documentation.
sbt clean compile
To create assemblies:
To build a particular project:
sbt "project cassandra" compile
To test a particular project:
sbt "project cassandra" test
To create a jar of a particular project:
sbt "project cassandra" assembly
Running E2E tests
If not already built, you must first build the connector archives:
sbt "project cassandra" assembly sbt "project elastic6" assembly sbt "project mongodb" assembly sbt "project redis" assembly
To run the tests:
We'd love to accept your contributions! Please use GitHub pull requests: fork the repo, develop and test your code, semantically commit and submit a pull request. Thanks!