ADAM is a genomics analysis platform with specialized file formats built using Apache Avro, Apache Spark, and Apache Parquet. Apache 2 licensed.

[PROJECT IS NO LONGER MAINTAINED] Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.

Avro schema generation and serialization / deserialization for Scala

A tool for data sampling, data generation, and data diffing

Avro SerDe for Apache Spark structured APIs.

Generate Scala case class definitions from Avro schemas

A collection of Magnolia add-on modules

Scala library to eliminate boilerplate

sbt plugin for generating Scala sources for Apache Avro schemas and protocols.

Lego bricks to build Apache Kafka serializers and deserializers

Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.

Functional Avro for Scala

Kafka Connect Sink/Source for Twitter

GCS support for avro-tools, parquet-tools and protobuf

Shapeless utilities for common data types

Examples of Avro, Kafka, Schema Registry, Kafka Streams, Interactive Queries, KSQL, Kafka Connect in Scala

Implicitly converts typeclass encoders to kafka Serializer, Deserializer, Serde.

Standalone alternatives to Kafka Connect Connectors

A light Kafka to HDFS/S3 ETL library based on Apache Spark

Convert XSD -> AVSC and XML -> AVRO