DO NOT MERGE: ES-Hadoop on Spark 2.0

[jamesblau]

This pull request builds elasticsearch-spark with Spark 2.0.0-SNAPSHOT, also using Scala 2.11 by default. Hopefully it would be ready for the Spark 2.0 release.

There are currently errors when building with Scala 2.10 (running ./gradlew -Pscala=210): it doesn't seem to be using the proper Spark version.

A sub-project elasticsearch-spark-1.3 replaces elasticsearch-spark-1.2. I also dropped the allProjects.configurations.all.resolutionStrategy block from build.gradle, which had seemed to cause the integrationTest task to hang indefinitely for this sub-project, although I don't know why.

(CLA pending, but will be approved before this is needed.)

[costin]

@jamesblau I appreciate the PR but what is it its purpose? Spark 2.0 is currently SNAPSHOT so why build against it?
Also since the PR has errors, why raise it in the first place? Why not wait for Spark 2.0?

Fwiw, there's already a spark 2.0 branch [1] that I've created some time ago; I haven't kept it up to date or pushed it since things are too much in flux.

[1] https://github.com/costin/elasticsearch-hadoop/tree/spark20

[jurriaan]

@jamesblau Thanks a lot, this is quite useful! Now running against a custom build using your changes so I can use Spark 2.0.

[jmritz]

@costin Is there any status on Spark 2.0.0 integration?

[costin]

Yes, it should land in master later today. We're currently testing the PR from @jimferenczi which increases the parallelism as well (see the master).

[costin]

Since support for Spark 2.0 is close to be added in master and this PR is old, I'm closing it down.
Thanks for the contribution and next time, please consider signing the CLA.

Cheer,s

[costin]

@jmritz Spark 2.0 just hit master. See #647