[PROJECT IS NO LONGER MAINTAINED] Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.

ADAM is a genomics analysis platform with specialized file formats built using Apache Avro, Apache Spark, and Apache Parquet. Apache 2 licensed.

Avro schema generation and serialization / deserialization for Scala

Generate Scala case class definitions from Avro schemas

Avro SerDe for Apache Spark structured APIs.

A tool for data sampling, data generation, and data diffing

sbt plugin for generating Scala sources for Apache Avro schemas and protocols.

Kafka Connect Sink/Source for Twitter

Functional Avro for Scala

Convert XSD -> AVSC and XML -> AVRO

A collection of Magnolia add-on modules

A light Kafka to HDFS/S3 ETL library based on Apache Spark

Lego bricks to build Apache Kafka serializers and deserializers

Examples of Avro, Kafka, Schema Registry, Kafka Streams, Interactive Queries, KSQL, Kafka Connect in Scala

GCS support for avro-tools, parquet-tools and protobuf

Scala library to eliminate boilerplate

Standalone alternatives to Kafka Connect Connectors

Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.

Spark UDFs to deserialize Avro messages with schemas stored in Schema Registry.

Exasol Cloud Storage Extension for accessing formatted data Avro, Orc and Parquet, on public cloud storage systems