Skip to content
Fast Apache Avro serialization/deserialization library
Java
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
src added support for "avro.java.string" property May 27, 2019
.gitignore performance tests improvements Nov 16, 2018
.travis.yml Updated travis jdk Sep 6, 2019
LICENSE
README.md 1.0.5 release May 27, 2019
pom.xml 1.0.5 release May 27, 2019

README.md

avro-fastserde

avro-fastserde is an alternative approach to Apache Avro serialization and deserialization. It generates dedicated code responsible for handling serialization and deserialization, which achieves better performance results than native implementation. Learn more here.

build status

Version

Current version is 1.0.5

Requirements

You need Java 8 to use this library.

Installation

Releases are distributed on Maven central:

<dependency>
    <groupId>com.rtbhouse</groupId>
    <artifactId>avro-fastserde</artifactId>
    <version>1.0.5</version>
</dependency>

Usage

Just use avro-fastserde DatumReader and DatumWriter interface implementation:

import com.rtbhouse.utils.avro.FastGenericDatumReader;
import com.rtbhouse.utils.avro.FastGenericDatumWriter;
import com.rtbhouse.utils.avro.FastSpecificDatumReader;
import com.rtbhouse.utils.avro.FastSpecificDatumWriter;

...

FastGenericDatumReader<GenericData.Record> fastGenericDatumReader = new FastGenericDatumReader<>(writerSchema, readerSchema);
fastGenericDatumReader.read(null, binaryDecoder);

FastGenericDatumWriter<GenericData.Record> fastGenericDatumWriter = new FastGenericDatumWriter<>(schema);
fastGenericDatumWriter.read(data, binaryEncoder);

FastSpecificDatumReader<T> fastSpecificDatumReader = new FastSpecificDatumReader<>(writerSchema, readerSchema);
fastSpecificDatumReader.read(null, binaryDecoder);

FastSpecificDatumWriter<T> fastSpecificDatumWriter = new FastSpecificDatumWriter<>(schema);
fastSpecificDatumWriter.write(data, binaryEncoder);

You can alter class generation behaviour via system properties:

 // Set compilation classpath
 System.setProperty(FastSerdeCache.CLASSPATH, compileClasspath);
 
 // Set generated classes directory
 System.setProperty(FastSerdeCache.GENERATED_CLASSES_DIR, generatedClassesDir);

Or FastSerdeCache class:

import com.rtbhouse.utils.avro.FastGenericDatumReader;
import com.rtbhouse.utils.avro.FastSerdeCache;

...

FastSerdeCache cache = new FastSerdeCache(compileClassPath);
FastGenericDatumReader<GenericData.Record> fastGenericDatumReader = new FastGenericDatumReader<>(writerSchema, readerSchema, cache);

Limitations

  • no support for reuse parameter in DatumReader interface.
  • no support for SchemaConstructable marker interface for specific Avro records.
  • FastSpecificDatumReader will not read data into GenericRecord if the specific classes are not available but will result in compilation failure and fall back to default SpecificDatumReader implementation.
You can’t perform that action at this time.