Streaming reference architecture built around Kafka.
A collection of components to build a real time ingestion pipeline
Kafka connect Cassandra connector and sink task to write Kafka topic payloads to Cassandra.
Kafka connect Elastic Search connector and sink task to write Kafka topic payloads to Elastic Search.
Kafka connect Kudu connector and sink task to write Kafka topic payloads to Kudu.
Kafka connect Bloomberg connector and source task to subscribe to Bloomberg streams via the open API and write to Kafka.
Kafka connect Druid connect and sink to write Kafka topic payloads to Druid.
WORK IN PROGRESS
- HBase
- OrientDb
- Neo4j
- Rest (spray-can post)
- UPC Scada Source and Sink
- MQTT Sink
- RDBMS (Sql Server 2016 supports JSON inserts)
- TCP sockets?
- Solr