Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-33046][DOCS] Update how to build doc for Scala 2.13 with sbt #29921

Closed
wants to merge 2 commits into from
Closed

[SPARK-33046][DOCS] Update how to build doc for Scala 2.13 with sbt #29921

wants to merge 2 commits into from

Conversation

sarutak
Copy link
Member

@sarutak sarutak commented Oct 1, 2020 •
edited
Loading

What changes were proposed in this pull request?

This PR fixes the description how to build Spark for Scala 2.13 with sbt.
In the current doc, how to build Spark for Scala 2.13 with sbt is described like:
scala-2 13-build-before

But build fails with this command because scala-2.13 profile is not enabled and scala-parallel-collections is absent.

[error] /home/kou/work/oss/spark-scala-2.13/core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala:23: object parallel is not a member of package collection

The correct command should be:

build/sbt -Pspark-2.13 compile

Why are the changes needed?

The build command is wrong.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

I checked that sbt -Pspark-2.13 is correct with the following command:

build/sbt -Dscala.version=2.13.3 -Phive -Phive-thriftserver -Pyarn -Pkubernetes  compile

I also build the modified doc and checked the generated html:
spark-scala-2 13-build-doc-after

@SparkQA
Copy link

SparkQA commented Oct 1, 2020

Test build #129306 has finished for PR 29921 at commit 0d06054.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Oct 1, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33921/

@SparkQA
Copy link

SparkQA commented Oct 1, 2020

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33921/

@sarutak
Copy link
Member Author

sarutak commented Oct 1, 2020

retest this please.

@SparkQA
Copy link

SparkQA commented Oct 1, 2020

Test build #129307 has finished for PR 29921 at commit 0d06054.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Oct 1, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33922/

@SparkQA
Copy link

SparkQA commented Oct 1, 2020

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33922/

dongjoon-hyun
dongjoon-hyun reviewed Oct 1, 2020
View reviewed changes
docs/building-spark.md Outdated
@@ -265,15 +265,13 @@ Change the major Scala version using (e.g. 2.13):

./dev/change-scala-version.sh 2.13

For Maven, please enable the profile (e.g. 2.13):
Eenable the profile (e.g. 2.13):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sarutak . nit. A typo: Eenable -> Enable?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I'll fix it. Thanks.

dongjoon-hyun
dongjoon-hyun approved these changes Oct 1, 2020
View reviewed changes
Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM (except one typo)

@dongjoon-hyun dongjoon-hyun changed the title [SPARK-33046][DOCS] How to build for Scala 2.13 with sbt in the doc is wrong. [SPARK-33046][DOCS] Update how to build doc for Scala 2.13 with sbt Oct 1, 2020
@SparkQA
Copy link

SparkQA commented Oct 1, 2020

Test build #129321 has finished for PR 29921 at commit 4c712f3.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@srowen srowen closed this in 0059997 Oct 1, 2020
@srowen
Copy link
Member

srowen commented Oct 1, 2020

Merged to master

@SparkQA
Copy link

SparkQA commented Oct 1, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33934/

@SparkQA
Copy link

SparkQA commented Oct 1, 2020

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33934/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@srowen srowen srowen approved these changes

@dongjoon-hyun dongjoon-hyun dongjoon-hyun approved these changes

Assignees
No one assigned
Labels
DOCS
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@sarutak @SparkQA @srowen @dongjoon-hyun