Cassandra connector for AWS Kinesis
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
src
.gitignore
.travis.yml
LICENSE
README.md
checkstyle-suppressions.xml
checkstyle.properties
checkstyle.xml
pom.xml
smartcat-formatter.xml

README.md

Cassandra Kinesis Connector

Cassandra connector for AWS Kinesis that allows Kinesis records to be stored into Cassandra.

Introduction

Cassandra Kinesis connector is based on Amazon Kinesis Connector Library project. It extends the library with Cassandra-specific transformer and emitter (ITransformer and IEmitter, that is, CassandraTransformer and CassandraEmitter) and provides basic implementations for those. It also includes a connector executor implementation that allows the connector to be run as a stand-alone Java process.

The provided default implementations of CassandraTransformer and CassandraEmitter are JsonCassandraTransformer and DefaultCassandraEmitter.

JsonCassandraTransformer interprets the input records from Kinesis stream as JSON strings and converts them into CassandraRecord instances. Every JSON top-level property is treated as a Cassandra table column. Only JSON properties of type string, boolean or number are processed. The rest is silently ignored.

DefaultCassandraEmitter stores CassandraRecord instances (produced by JsonCassandraTransformer) into the respective Cassandra table (specified by the configuration).

Configuration

Cassandra Kinesis Connector extends Amazon Kinesis Connector Library's configuration and adds additional Cassandra-specific properties.

All default property values can be replaced using an external properties file - cassandra-kinesis-connector.properties.

Usage

Cassandra Kinesis Conector can be either embedded as a library into an application or used as a out-of-the-box connector application.

For embedding, see CassandraKinesisConnectorExecutor implementation.

As a stand-alone, out-of-the-box connector application, Cassandra Kinesis Connector executable JAR (cassandra-kinesis-VERSION-all.jar) should be used. When started, the connector tries to load external properties file cassandra-kinesis-connector.properties from the working directory.

License and development

Cassandra Kinesis is licensed under the liberal and business-friendly Apache Licence, Version 2.0 and is freely available on GitHub. Cassandra Kinesis is further released to the repositories of Maven Central and on JCenter. The project is built using Maven. From your shell, cloning and building the project would go something like this:

git clone https://github.com/smartcat-labs/cassandra-kinesis.git
cd cassandra-kinesis
mvn package