Unit Test MapReduce Jobs using LocalJobRunner
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
hhdp-core
.gitignore
README.md
build.gradle
settings.gradle

README.md

hhdp

Test MapReduce Jobs using LocalJobRunner

Get Started

Depend on the hadoop version you use, you can either implement the interface me.haogao.hhdp.job.mapreduce.JobModule or me.haogao.hhdp.job.mapred.JobModule
Read some test cases under hhdp-core/src/test/ can be a good start

Running Tests

gradle clean test
gradle test --tests me.haogao.hhdp.test.mapred.driver.TestJobDriver.testPassThoughJob

To Do

  1. Ability to test workflow
  2. Example on combiner
  3. Example on InputFormat, OutputFormat
  4. miniCluster