Skip to content

Giraph use your own version of VertexInputFormat

Cindyxinxin edited this page Aug 28, 2014 · 1 revision

Assume you want to use JsonDoubleDoubleFloatDoubleVertexInputFormat and the .java file is saved in /myhome/giraph-work

Step1:

###copy the .java file into the corrent directory (In our docker, the java files for the formats are saved in /usr/local/giraph/giraph-core/src/main/java/org/apache/giraph/io/formats)

cp /myhome/giraph-work/JsonDoubleDoubleFloatDoubleVertexInputFormat.java /usr/local/giraph/giraph-core/src/main/java/org/apache/giraph/io/formats

Step2:

cd into the directory and compile the .java file with the built in jar file from Giraph

  1. cd /usr/local/giraph/giraph-core/src/main/java/org/apache/giraph/io/formats
  2. javac -cp /usr/local/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.4.1-jar-with-dependencies.jar:$($HADOOP_HOME/bin/hadoop classpath) JsonDoubleDoubleFloatDoubleVertexInputFormat.java

If compiled successed, you will see the .class files.

In this example, you can see JsonDoubleDoubleFloatDoubleVertexInputFormat.class and JsonDoubleDoubleFloatDoubleVertexInputFormat$JsonDoubleDoubleFloatDoubleVertexReader.class since there are two classes in the java file.

Step3:

In order to put the two .class files to the same directory (org/apache/giraph/io/formats/) as the other .class files, cd into the parent directory of "org" (/usr/local/giraph/giraph-core/src/main/java/).

cd /usr/local/giraph/giraph-core/src/main/java/

Then update the jar file by typing:

jar uf /usr/local/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.4.1-jar-with-dependencies.jar org/apache/giraph/io/formats/JsonDoubleDoubleFloatDoubleVertexInputFormat.class org/apache/giraph/io/formats/JsonDoubleDoubleFloatDoubleVertexInputFormat$JsonDoubleDoubleFloatDoubleVertexReader.class

Step4:

copy the updated jar file to /myhome/giraph-work in order to reuse next time

cp /usr/local/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.4.1-jar-with-dependencies.jar /myhome/giraph-work/

Step5:

cd into /myhome/giraph-work/ and follow the instruction "Compiling and running your own Giraph code" (https://github.com/uwsampa/giraph-docker). Remember to use the jar file in /myhome/giraph-work/ directory if the jar file in the original directory is not updated when you restart the docker