This project is intended to assist in the automatic generation of FlatBuffer schemas via Java annotations in source.
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
aurkitu-annotations
aurkitu-maven-plugin
aurkitu-test-dependency
aurkitu-test-service
.codecov.yml
.gitignore
.travis.yml
LICENSE.md
README.md
package.sh
pom.xml

README.md

aurkitu

Build Status codecov Maven Central

Aurkitu is the Basque word for for the English phrase "to find". This project is intended to assist in the automatic generation of FlatBuffer IDL/schemas from Java source. But isn't this backwards? Shouldn't the schema generate the models for both client and service? Unfortunately, when using models which are stored in cloud based storage mechanisms (e.g. AWS Dynamo DB, annotations, etc.) requires that table definitions, keys, and other attributes to be declared within the model. Hence with this project the aim is to allow the server implementation to remain as the master version of these models making the schema, flatbuffers, and client code as auxillery.

Aurkitu currently supports Flatbuffers version 1.3.

please note

This is a very early proof-of-concept currently being developed in spare time.

roadmap

  • handle core types
  • handle options, deprecation, and default values
  • validate dependencies (optional)
  • implement as plugin and test against a sample project
  • test with flatc (started)
  • update with current (1.8) feature support (i.e. gRPC, Field, String constant, etc.)
  • release to maven
  • split into annotations and plugin

peculiarities

While flatbuffers support unsigned primatives (e.g. ubyte, ushort, etc.), Java does not technically support them (though you can use the wrapper types [e.g. java.lang.Long, etc.]) to simulate this behavior. Eventually we could map to the wrapper types when building the schema files. At this time all primatives and their corresponding wrappers are mapped as primatives.

Aurkitu supports java.util.Map by creating an autogenerated key-value type which is then used via list in the defining type. It is a work around and the parsers using generated flatbuffers will need to translate accordingly. The name format is MapValueSet_<type>_<field>. The server and client will need to implement custom code to translate back and forth between Object{K, V}[] and Map<K, V>.

integration

Annotations

Add the following to your dependencies within your pom.xml:

<dependency>
    <groupId>com.michaelhradek</groupId>
    <artifactId>aurkitu-annotations</artifactId>
    <version>0.0.5.1</version>
</dependency>

Maven Plugin

Followed by the following to the plugins of your build specifications within your pom.xml:

<plugin>
    <groupId>com.michaelhradek</groupId>
    <artifactId>aurkitu-maven-plugin</artifactId>
    <version>0.0.5.1</version>
    <configuration>
        <schemaName>user</schemaName>
        <schemaNamespace>com.company.package.subpackage.flatbuffers</schemaNamespace>
        <specifiedDependencies>
             <depdendency>com.company.department:artifact-id</depdendency>
             <depdendency>com.other.subteam</depdendency>
        </specifiedDependencies>
        <consolidatedSchemas>true</consolidatedSchemas>
        <schemaIncludes>
             <include>"../../../../target/maven-shared-archive-resources/flatbuffers/other.fbs"</include>
        </schemaIncludes>
        <validateSchema>true</validateSchema>
        <namespaceOverrideMap>
	          <com.company.package.search>com.company.package.replace</com.company.package.search>
        </namespaceOverrideMap>   
        <useSchemaCaching>false</useSchemaCaching>     
    </configuration>
    <executions>
        <execution>
            <phase>process-classes</phase>
            <goals>
                <goal>build-schema</goal>
            </goals>
        </execution>
    </executions>
</plugin>

option definitions

required

  • schemaName: sets the name of the generated schema which is then used for the output filename (e.g. <schemaName>.<flatcFilename> or myschema.fbs)

optional

  • schemaNamespace: sets the namespace of schema. All objects in this schema will have this namespace (default: generated.flatbuffers)
  • flatcExtention: sets the output file extension (default fbs)
  • schemaFileIdentifier: flatbuffer file identifier (e.g. file_identifier "MYFI";)
  • outputDir: where the generated schema will be written to (default: ${project.build.directory}/aurkitu/schemas)
  • validateSchema: if true, validate the schema and append the report to the end of the schema file as a list of comments (default: true)
  • generateVersion: if true, generate a version for the schema which excludes the validation text and then add it as a comment to the top of the schema file (default: false)
  • useSchemaCaching: if true, skip schema generation. The file that is located in the output directory will be used. Designed for local development build speed improvement (default: false)
  • namespaceOverrideMap: allows for schema namespaces to be overriden. This is handy when using includes and schemas from other projects (e.g <com.company.package.search>com.company.package.replace</com.company.package.search>)
  • schemaIncludes: allows for configuration of schema includes (e.g. <include>"../../../../target/maven-shared-archive-resources/flatbuffers/other.fbs"</include>)
  • specifiedDependencies: allows for configuration of targeted dependency searching for specific dependencies for annotations. If this is specified, a artifact resolution will be kept to a minimum greatly increasing build speed. You can specify any number of packages. If you specify the groupId only then the entirety of the group will be included in artifact resolution (e.g. <dependency>com.company.group</dependency>) . To specify a specific artifact use groupId:artifactId (e.g. <dependency>com.company.group:artifact</dependency>)

planned

  • consolidatedSchemas: if true, create one schema. If false, create one schema for the project and then one schema per dependecy. (default: true) This is useful in situations where the dependencies are used across projects where namespaces are useful.

usage

Through the use of annotations:

@FlatBufferTable
public class SampleClassReferenced {

    @FlatBufferFieldOptions(fieldType = FieldType.IDENT, useFullName = true)
    SampleClassReferenced fullnameClass;
    
    @FlatBufferFieldOptions(fieldType = FieldType.IDENT, defaultValue = "SHORT_SWORD")
    protected SampleClassTableInnerEnumInt innerEnum;
...

Specify a root type:

@FlatBufferTable(rootType = true)
public class SampleClassTable {
...

Specify different structure declarations:

@FlatBufferTable(TableStructureType.STRUCT)
public class SampleClassStruct {
...
}

@FlatBufferEnum(enumType = FieldType.BYTE)
public enum SampleEnumByte {

    // If you specify a enumType you will need to specify which field represents the type (as there can be several fields within an `ENUM`)
    @FlatBufferEnumTypeField
    byte id;
    
    String description;
    
    SampleEnumByte(byte id, String description) {
...

Ignore fields:

@FlatBufferIgnore
protected String ignore;

sample output

This sample is generated by the JUnit tests within the project into a test.fbs file.

// Aurkitu automatically generated IDL FlatBuffer Schema
// @version: c4f82c4d

// This is a enum comment
enum VectorSize : short { SMALL = 10000, MEDIUM = 20000, LARGE = 30000 }

enum SampleEnumNull : byte { PlatformAlpha, PlatformBeta, PlatformGamma }

enum Matrix : int { SMALL = 10000, MEDIUM = 20000, LARGE = 30000 }

enum SplineEstimate : long { SMALL = 10000, MEDIUM = 20000, LARGE = 30000 }

enum Option { FirstOption, SecondOption, ThirdOption }

enum SampleClassTableInnerEnumInt { DAGGER, SHORT_SWORD, SWORD, GREAT_SWORD }

enum SampleEnumByte : byte { EnvironmentAlpha = 1, EnvironmentBeta = 2, EnvironmentGamma = 3 }

struct SampleClassStruct {
  x:float;
  y:float;
  z:float;
}

// Auto-generated type for use with Map<?, ?>
table MapValueSet_SampleClassTable_dataMap {
  key:string;
  value:string;
}

// This is a type level comment
table SampleClassTable {
  id:long;
  name:string;
  level:short;	// This is a field level comment
  currency:int;
  createTime:long;
  tokens:[string];
  deleted:bool;
  energy:byte;
  weight:double;
  options:[int];
  anomalousSamples:[SimpleUndefinedClass];
  dataMap:[MapValueSet_SampleClassTable_dataMap];
  definedInnerEnumArray:[SampleClassReferenced$SampleClassTableInnerEnumInt];
  innerEnum:SampleClassTableInnerEnumInt = SHORT_SWORD;
  fullnameClass:SampleClassReferenced;
}

table SampleAnonymousEnum {
  option:Option;
  size:VectorSize;
  estimate:SplineEstimate;
  matrix:Matrix = SMALL;
}

table SampleClassTableWithUndefined {
  id:long;
  message:string;
  awesomeUndefinedClass:SimpleUndefinedClass;
}

table SampleClassReferenced {
  id:long;
  baggage:[SampleClassTable];
  samples:[SampleClassNamespaceMap];
  abstractField:string;
}

table InnerClassStatic {
  virulant:bool;
}

table InnerClass {
  processed:bool;
  weaponType:SampleClassTableInnerEnumInt;
}

table SampleClassNamespaceMap {
  id:string;
}

root_type SampleClassTable;

// Schema failed validation (i.e. flatc will likely fail): 
// Issue : TYPE_DEFINITION_NOT_DEFINED, Location: SampleClassTable, Name: anomalousSamples
// Issue : INVALID_PATH, Location: SampleClassTable, Name: definedInnerEnumArray, Comment: Array type name contains '$'; using '@FlatBufferOptions(useFullName = true)' on inner not recommended: SampleClassReferenced$SampleClassTableInnerEnumInt
// Issue : TYPE_DEFINITION_NOT_DEFINED, Location: SampleClassTableWithUndefined, Name: awesomeUndefinedClass
// Issue : MISCONFIGURED_DEFINITION, Location: Option, Name: null
// Issue : MISCONFIGURED_DEFINITION, Location: SampleClassTableInnerEnumInt, Name: null

With schema validation enabled, comments will be added to the end of the generated schema file which should assist in the resolution of issues which may likely cause flatc to fail.