A Pig to JSON UDF for Pig that converts tuples and bags to JSON strings
Java
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
dist/lib Bugfix Sep 26, 2012
src/com/hortonworks/pig/udf
.gitignore
Pig-to-json.iml
README.md
build.xml
ivy.xml
test.pig

README.md

pig-to-json

A Pig to JSON UDF for Pig that converts tuples and bags to JSON strings. This code is free under the Apache 2.0 license.

This project likely borrows (I'm not sure, its been a while since I started it and I looked at lots of stuff) from the following projects:

https://github.com/danharvey/pigJsonUtils http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/builtin/JsonStorage.java https://github.com/Ganglion/sounder/blob/master/udf/src/main/java/sounder/pig/json/ToJson.java

Building the Project

ant clean
ant dist

Using the ToJson UDF

The file test.pig is illustrative:

/* Load Avro jars and define shortcut */
register /me/Software/pig/build/ivy/lib/Pig/avro-1.5.3.jar
register /me/Software/pig/build/ivy/lib/Pig/json-simple-1.1.jar
register /me/Software/pig/contrib/piggybank/java/piggybank.jar
define AvroStorage org.apache.pig.piggybank.storage.avro.AvroStorage();

register /me/Software/pig-to-json/dist/lib/pig-to-json.jar

-- Available at https://s3.amazonaws.com/rjurney_public_web/hadoop/enron.avro
emails = load '/me/Data/enron.avro' using AvroStorage();
emails = limit emails 10;
json_test = foreach emails generate message_id, com.hortonworks.pig.udf.ToJson(tos) as bag_json;
dump json_test

emails2 = load '/me/Data/enron.avro' using AvroStorage();
emails2 = limit emails2 10;
json_test2 = foreach emails2 generate message_id, com.hortonworks.pig.udf.ToJson(from) as tuple_json;
dump json_test2