[SPARK-10556] Remove explicit Scala version for sbt project build files

[ahirreddy]

Previously, project/plugins.sbt explicitly set scalaVersion to 2.10.4. This can cause issues when using a version of sbt that is compiled against a different version of Scala (for example sbt 0.13.9 uses 2.10.5). Removing this explicit setting will cause build files to be compiled and run against the same version of Scala that sbt is compiled against.

Note that this only applies to the project build files (items in project/), it is distinct from the version of Scala we target for the actual spark compilation.

[vanzin]

Removing this explict setting will cause build files to be compiled and run against the same version of Scala that sbt is compiled against.

I'm not too sure that's desirable, though.

[marmbrus]

@vanzin I believe that is what SBT expects, I'm not sure why this setting was there in the first place. You shouldn't be running files compiled with different versions of scala together or you can get weird abstract method errors.

[ahirreddy]

The problem with explicitly setting a different Scala version for build files is that SBT expects plugins and build files to be compiled against the same version of Scala it is running on, otherwise it's possible to run into errors (NoSuchMethodError or ClassNotFoundException) that arise from changes in Scala.

[vanzin]

I'm not doubting that, just questioning the fact that we're now allowing people to build Spark against "non-blessed" versions of Scala. Which might be ok, just feels a little weird.

(The alternative would be to fail if the version is not the expected.)

[marmbrus]

To clarify, this is not changing how Spark is built. It's changing how project/SparkBuild.scala is built, which in turn is used to compile spark when someone is using SBT. project/SparkBuild.scala will have its own notion of scalaVersion that will be used to compile Spark.

[vanzin]

Ah, ok. That clarifies things. In that case, move on, nothing to see here. :-)

[SparkQA]

Test build #42305 has finished for PR 8709 at commit f17ee06.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #1740 has finished for PR 8709 at commit f17ee06.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #1741 has finished for PR 8709 at commit f17ee06.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[srowen]

Merged into master / 1.5 / 1.4 / 1.3