From 65d84ec13ec57787a728338c633f5f262b0decb8 Mon Sep 17 00:00:00 2001 From: Prashant Sharma Date: Tue, 18 Nov 2014 12:54:38 +0530 Subject: [PATCH 1/3] Update building-spark.md --- docs/building-spark.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/docs/building-spark.md b/docs/building-spark.md index bb18414092aae..7b7b6d91e40d9 100644 --- a/docs/building-spark.md +++ b/docs/building-spark.md @@ -113,7 +113,17 @@ mvn -Pyarn -Phive -Phive-thriftserver-0.12.0 -Phadoop-2.4 -Dhadoop.version=2.4.0 {% endhighlight %} # Building for Scala 2.11 -To produce a Spark package compiled with Scala 2.11, use the `-Dscala-2.11` property: +### For SBT users just start the console as + +`sbt/sbt -Pscala-2.11` + +And then run the commands as usual, e.g. clean, assembly or test etc.. + +### For maven users it is a two step process. + +Step 1: First run `dev/change-version-to-2.11.sh` script from `$SPARK_HOME`. + +Step 2: Produce a Spark package compiled with Scala 2.11, use the `-Dscala-2.11` property: mvn -Pyarn -Phadoop-2.4 -Dscala-2.11 -DskipTests clean package From ebf86841673dac4ef3be1af3006a751d68da9842 Mon Sep 17 00:00:00 2001 From: Prashant Sharma Date: Tue, 18 Nov 2014 16:40:05 +0530 Subject: [PATCH 2/3] Fixed spark build and added new build instructions. --- docs/building-spark.md | 2 +- project/SparkBuild.scala | 10 +--------- 2 files changed, 2 insertions(+), 10 deletions(-) diff --git a/docs/building-spark.md b/docs/building-spark.md index 7b7b6d91e40d9..b9c11b2f35da6 100644 --- a/docs/building-spark.md +++ b/docs/building-spark.md @@ -115,7 +115,7 @@ mvn -Pyarn -Phive -Phive-thriftserver-0.12.0 -Phadoop-2.4 -Dhadoop.version=2.4.0 # Building for Scala 2.11 ### For SBT users just start the console as -`sbt/sbt -Pscala-2.11` +`sbt/sbt -Dscala-2.11="1"` And then run the commands as usual, e.g. clean, assembly or test etc.. diff --git a/project/SparkBuild.scala b/project/SparkBuild.scala index 1697b6d4f2d43..8cc7ab3eb49e9 100644 --- a/project/SparkBuild.scala +++ b/project/SparkBuild.scala @@ -100,15 +100,7 @@ object SparkBuild extends PomBuild { "conjunction with environment variable.") v.split("(\\s+|,)").filterNot(_.isEmpty).map(_.trim.replaceAll("-P", "")).toSeq } - - if (profiles.exists(_.contains("scala-"))) { - profiles - } else if (System.getProperty("scala-2.11") != null) { - profiles ++ Seq("scala-2.11") - } else { - println("Enabled default scala profile") - profiles ++ Seq("scala-2.10") - } + profiles } Properties.envOrNone("SBT_MAVEN_PROPERTIES") match { From ac7b9dfc96d74740b44d9641ff975470d8bbffc9 Mon Sep 17 00:00:00 2001 From: Prashant Sharma Date: Wed, 19 Nov 2014 12:00:18 +0530 Subject: [PATCH 3/3] Update building-spark.md --- docs/building-spark.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/building-spark.md b/docs/building-spark.md index b9c11b2f35da6..bd4a23bad365a 100644 --- a/docs/building-spark.md +++ b/docs/building-spark.md @@ -121,9 +121,9 @@ And then run the commands as usual, e.g. clean, assembly or test etc.. ### For maven users it is a two step process. -Step 1: First run `dev/change-version-to-2.11.sh` script from `$SPARK_HOME`. +Step 1: First run `dev/change-version-to-2.11.sh`. -Step 2: Produce a Spark package compiled with Scala 2.11, use the `-Dscala-2.11` property: +Step 2: To produce a Spark package compiled with Scala 2.11, use the `-Dscala-2.11` property: mvn -Pyarn -Phadoop-2.4 -Dscala-2.11 -DskipTests clean package