Skip to content
Protocol Buffer Scheme for Cascading
Java
Find file
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.
src
.gitignore
README.md
pom.xml

README.md

cascading.protobuf

Provides a ProtobufSequenceFileScheme suitable for consuming Protocol Buffer encoded messages. Currently only supports operation as a source.

Installing

cascading.protobuf is hosted on conjars.

Leiningen

[cascading.protobuf "0.0.3"]

Maven

<dependency>
  <groupId>cascading.protobuf</groupId>
  <artifactId>cascading.protobuf</artifactId>
  <version>0.0.3</version>
</dependency>

Usage

The scheme assumes that the input to your source is a SequenceFile<LongWritable, BytesWritable> where the value contains the raw serialized bytes of your message.

Because Protocol Buffers are schema-based, when constructing the scheme it's also necessary to provide the generated Java class for your message:

String inputPath = "./input.seq";
Scheme personScheme = new ProtobufSequenceFileScheme(Messages.Person.class, new Fields("id", "name", "email"))
Tap source = new Lfs(scheme, inputPath);
Something went wrong with that request. Please try again.