Skip to content

Utility framework for composing Avro Schemas. From smaller components (types) specified in separate files into large schemas ready to be deployed to schema registry or your application

License

michalklempa/avro-compose

develop
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 

Table of Contents generated with DocToc

Avro Compose

Utility framework for composing Avro Schemas. From smaller components (types) specified in separate files into large schemas ready to be deployed to schema registry or your application.

Requirements

Motivation

With groving number of schema files, one quickly finds out, that decomposing the schemas into smaller components (types) is needed. With schema definition language (the JSON), there is no possibility to import other schemas into current file. When we do not want to rewrite schema into Avro Interface Definition Language as proposed in 1, we have two options to ensure correct ordering of input files before parsing (either by avro-maven-plugin or our own code):

  1. name the files so they order lexicographically and ensure proper loading
  2. write our own tool to parse the files and match dependencies (= this project)

In this project we do not require any particular ordering of input files. If they can parse as a whole, we find correct ordering programatically. All you need is to provide the input schemas.

Download

Simple JAR

To download the artifact run:

mvn dependency:copy -Dartifact=com.michalklempa:avro-compose:0.0.1:jar:shaded -DoutputDirectory=.

For a simple example with composing schemas, look into example-simple

Maven setup

You may incorporate the artifact into your Java project as a dependency and then run the schema generation as a part of project build:

        <dependency>
            <groupId>com.michalklempa</groupId>
            <artifactId>avro-compose</artifactId>
            <version>0.0.1</version>
            <type>jar</type>
        </dependency>

Full example project with pom.xml schema generation and even the pom.xml itself templated can be found in `example-project

References

[1] Björn Beskow: Serialization, Schema Compositionality and Apache Avro

About

Utility framework for composing Avro Schemas. From smaller components (types) specified in separate files into large schemas ready to be deployed to schema registry or your application

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages