Skip to content
A repository with Map Reduce examples in Hadoop 2 (YARN API)
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
nbproject
src
.gitignore
LICENSE
README.md
build.xml

README.md

YarnExamples

A repository with Map Reduce examples in Hadoop 2 (YARN API)

Examples at the moment:

How to execute the examples?

I assume which you clone this repository, you compiled and build a jar file with netbeans, and you have installed Hadoop 2.X.

If the before it's ok, the DgIndexer should be executed with the next command:

hadoop jar /path_to_jar_file/YarnExamples.jar dgIndexer.DgIndexer /user/hduser/data/TrecFile.txt /user/hduser/OutputFolder

If the example finish success, you'll can consulting the result:

hdfs dfs -cat /user/hduser/OutputFolder/part-r-00000

Tips

Set arbitrary reducers number: mapreduce.job.reduces

hadoop jar /path_to_jar_file/YarnExamples.jar dgIndexer.DgIndexer -Dmapreduce.job.reduces=2 /user/hduser/data/TrecFile.txt /user/hduser/OutputFolder
You can’t perform that action at this time.