Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Protocol Buffer Scheme for Cascading

branch: master

Fetching latest commit…

Octocat-spinner-32-eaf2f5

Cannot retrieve the latest commit at this time

Octocat-spinner-32 src
Octocat-spinner-32 .gitignore
Octocat-spinner-32 README.md
Octocat-spinner-32 pom.xml
README.md

cascading.protobuf

Provides a ProtobufSequenceFileScheme suitable for consuming Protocol Buffer encoded messages. Currently only supports operation as a source.

Installing

cascading.protobuf is hosted on conjars.

Leiningen

[cascading.protobuf "0.0.3"]

Maven

<dependency>
  <groupId>cascading.protobuf</groupId>
  <artifactId>cascading.protobuf</artifactId>
  <version>0.0.3</version>
</dependency>

Usage

The scheme assumes that the input to your source is a SequenceFile<LongWritable, BytesWritable> where the value contains the raw serialized bytes of your message.

Because Protocol Buffers are schema-based, when constructing the scheme it's also necessary to provide the generated Java class for your message:

String inputPath = "./input.seq";
Scheme personScheme = new ProtobufSequenceFileScheme(Messages.Person.class, new Fields("id", "name", "email"))
Tap source = new Lfs(scheme, inputPath);
Something went wrong with that request. Please try again.