mr-runner: add mapreduce-runner profile for examples, archetypes and …

[peihe]

…IT tests.

Follow this checklist to help us incorporate your contribution quickly and easily:

• Make sure there is a JIRA issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes.
• Each commit in the pull request should have a meaningful subject line and body.
• Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue.
• Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
• Run mvn clean verify to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

---

[peihe]

R: @kennknowles

[coveralls]

Changes Unknown when pulling 48c6dea on peihe:mr-runner-profile into ** on apache:mr-runner**.

[manuzhang]

@peihe Re-enable examples module in root pom ?

[peihe]

Done
added examples module back.

[peihe]

retest this please

[coveralls]

Changes Unknown when pulling 72ae520 on peihe:mr-runner-profile into ** on apache:mr-runner**.

[peihe]

retest this please

[coveralls]

Changes Unknown when pulling 72ae520 on peihe:mr-runner-profile into ** on apache:mr-runner**.

[manuzhang]

@peihe for pre-commit tests to work, mr-runner also has to be added to jenkins configuration file

[coveralls]

Changes Unknown when pulling 1819557 on peihe:mr-runner-profile into ** on apache:mr-runner**.

[kennknowles]

R: -@kennknowles
R: @manuzhang

(if that's OK - feel free to give back to me if you have no time; I have a bit more time this week)

[manuzhang]

@peihe

2017-09-18T03:32:04.718 [ERROR] testE2EWordCount(org.apache.beam.examples.WordCountIT)  Time elapsed: 0.148 s  <<< ERROR!
java.lang.IllegalArgumentException: Unknown 'runner' specified 'org.apache.beam.runners.mapreduce.MapReduceRunner', supported pipeline runners [ApexRunner, DataflowRunner, DirectRunner, FlinkRunner, SparkRunner, TestApexRunner, TestDataflowRunner, TestFlinkRunner, TestSparkRunner]
	at org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1615)
	at org.apache.beam.sdk.options.PipelineOptionsFactory.access$400(PipelineOptionsFactory.java:104)
	at org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:291)
	at org.apache.beam.sdk.testing.TestPipeline.testingPipelineOptions(TestPipeline.java:407)
	at org.apache.beam.examples.WordCountIT.testE2EWordCount(WordCountIT.java:59)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:393)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.beam.runners.mapreduce.MapReduceRunner
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1601)
	... 22 more

[peihe]

@jasonkuster @kennknowles

The mapreduce-runner profile was not activated in WordCountIT tests, even though I have updated ./job_beam_Java_UnitTest.groovy.

Is there other place to config Jenkins or do I need to submit Jenins change first?

[kennknowles]

Sorry for the slow reply. In fact the Jenkins jobs are built from the groovy files most recent on master. We don't have a way to do it differently, yet. That is https://issues.apache.org/jira/browse/BEAM-3047.

On this branch, you can add the runner to the jenkins-precommit profile. But quite soon we will move all of this to Gradle. Then you will need to change the gradle files on the branch.

[kennknowles]

Planning on wrapping this up?

[stale]

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@beam.apache.org list. Thank you for your contributions.

[stale]

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@beam.apache.org list. Thank you for your contributions.

[stale]

This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.