Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-35620][BUILD][PYTHON] Remove documentation build in Python linter #32760

Closed
wants to merge 1 commit into from

Conversation

HyukjinKwon
Copy link
Member

@HyukjinKwon HyukjinKwon commented Jun 3, 2021

What changes were proposed in this pull request?

This PR proposes to remove PySpark documentation build in linter check because:

  • to speed up CI build by removing duplicate documentation build (linter and doc build)
  • for [SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation #32726. With this PR PySpark documentation build requires a full Spark build to generate plot images in PySpark documentation. It makes less sense to require it in Python linter.
  • to remove unnecessary dependency installation for Python linter in CI

Why are the changes needed?

Python linter script includes documentation build. Because of this, we run documentation builds duplicately in CI, and requires unnecessary dependencies to be installed, and takes extra time. It would more make sense to exclude this in Python linter.

Does this PR introduce any user-facing change?

No, dev-only.

How was this patch tested?

Manually tested, and it will be tested in CI.

@HyukjinKwon
Copy link
Member Author

cc @dongjoon-hyun @srowen FYI

@itholic
Copy link
Contributor

itholic commented Jun 3, 2021

Looks fine to me if test passes.

Excluding the doc build from linter makes sense to me.

@SparkQA
Copy link

SparkQA commented Jun 3, 2021

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43781/

@HyukjinKwon
Copy link
Member Author

Thanks @itholic and @srowen.

Merged to master.

@SparkQA
Copy link

SparkQA commented Jun 3, 2021

Test build #139257 has finished for PR 32760 at commit ae2e5c5.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@dongjoon-hyun
Copy link
Member

+1, late LGTM. Thanks!

@HyukjinKwon HyukjinKwon deleted the SPARK-35620 branch January 4, 2022 00:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants