avro-kotlin

Avro Kotlin provides a type- and null-safe type-system that encapsulates the apache-java avro-types and makes them easily accessible from kotlin.

Since Version 1.11.4.2 we use a multi module build. The GroupId changed, best use the provided BOM:

<dependency>
    <groupId>io.toolisticon.kotlin.avro</groupId>
    <artifactId>avro-kotlin-bom</artifactId>
    <version>LATEST</version>
    <type>pom</type>
    <scope>import</scope>
</dependency>

Apache Avro

Apache Avro is a data serialization system.

Avro provides:

• Rich data structures.

• A compact, fast, binary data format.

• A container file, to store persistent data.

• Remote procedure call (RPC).

• Simple integration with dynamic languages. Code generation is not required to read or write data files nor to use or implement RPC protocols. Code generation as an optional optimization, only worth implementing for statically typed languages.

see: Avro Documentation

To simplify usage of the org.apache.avro:avro-core lib (written in java) from kotlin, this library wraps the java implementation. Parsing declarations and generating schema is still done using the underlying official library, but we built tools, wrappers and helpers on top.

why?

Working with avro files (reading, writing, copying, modifying, ...) can quickly become cumbersome, because the jvm file/resource io is far from convenience.

This lib provides simplified access to avro schema and protocol files by giving you fromResource, fromDirectory and writeToFile methods to read and write .avsc and .avpr files.

The assumption is, that the location of an avro file follows the java fqn/package convention, so when you have namespace=com.acme.foo and name=MySchema, the file/resource is expected under com/acme/foo/MySchema.avsc.

how?

• store your schema files according to the package convention
• Create a fqn wrapper schema(namepace="com.acme.foo", name="MySchema")
• use the extension functions fromResource or fromDirectory to load and parse the Schema. (Same applies to Protocols.)

Build

Semantic Versioning

To reflect the version of the underlying avro-core library, we use a 4-digit semantic versioning concept (semVer). Take the current release version: 1.11.2.0, which uses version 1.11.3 of the avro-core lib:

• The first two digits 1.11 refer to the major/minor digits of the apache lib. Together they build the major version of avro-kotlin. We assume our lib to not have breaking (major) changes, unless the underlying apache library provides them
• the last two digits 2.0 refer to the version of our library, where 2 indicates the second iteration when it comes to major features and 0 to the latest patch level, this version increases for (bug-)fixes mainly.

Releases

• all issues and dependabot updates must releate to a milestone. The milestone must be labeled with the intended release version. It is ok to change the milestone label before releasing, so a template name like 1.11.2.x is ok until you finally choose the minor/patch number.
• Before releasing: close the milestone (and finalize its label). Remember to create a new template milestone for follow-up issues.
  • Closing the milestone triggers a github action that creates a draft release with the version of the milestone label.
• Then, manually on your local console, run mvn gitflow:release-start which will create a release branch and interatively ask for the desired release version. It triggers a pipeline run that evaluates the release branch
• Finalize the release by running mvn gitflow:release-finish this will merge the release branch to master branch and thereby trigger the github release action that publishes the jar, sources and docs to sonatype/maven-central.
• Wait until the library is available on maven-central. This should take just a couple of minutes if you manually try to use it in a pom.xml via mvn -U (although it might take hours until it is indexed and the badge changes and dependabot starts to create requests).
• Go to the github releases and finalize the draft that was created in step 2.
• done