Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Protocol Buffer Scheme for Cascading
Java
branch: master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
src
.gitignore
README.md
pom.xml

README.md

cascading.protobuf

Provides a ProtobufSequenceFileScheme suitable for consuming Protocol Buffer encoded messages. Currently only supports operation as a source.

Installing

cascading.protobuf is hosted on conjars.

Leiningen

[cascading.protobuf "0.0.3"]

Maven

<dependency>
  <groupId>cascading.protobuf</groupId>
  <artifactId>cascading.protobuf</artifactId>
  <version>0.0.3</version>
</dependency>

Usage

The scheme assumes that the input to your source is a SequenceFile<LongWritable, BytesWritable> where the value contains the raw serialized bytes of your message.

Because Protocol Buffers are schema-based, when constructing the scheme it's also necessary to provide the generated Java class for your message:

String inputPath = "./input.seq";
Scheme personScheme = new ProtobufSequenceFileScheme(Messages.Person.class, new Fields("id", "name", "email"))
Tap source = new Lfs(scheme, inputPath);
Something went wrong with that request. Please try again.