[SPARK-57938][INFRA] Add new SKIP_ flags to doc build script#57010
[SPARK-57938][INFRA] Add new SKIP_ flags to doc build script#57010nchammas wants to merge 8 commits into
SKIP_ flags to doc build script#57010Conversation
|
cc @panbingkun, who authored the referenced PR. |
69b34c4 to
453ee24
Compare
453ee24 to
5e74966
Compare
| cd(SPARK_PROJECT_ROOT) | ||
|
|
||
| build_spark_scala_and_java_docs_if_necessary | ||
| if not (ENV['SKIP_UNIDOC'] == '1') |
There was a problem hiding this comment.
Please check possible latent inconsistency: SKIP_UNIDOC reuses a name that already exists cross-subsystem; dev/run-tests.py:307 reads it via truthiness and multiple non-doc CI jobs set SKIP_UNIDOC: true, whereas the doc script gates strictly on == '1'. No live conflict (the docs job at lines 1246-1577 sets neither new flag, so it builds both as before, and no CI path both exports SKIP_UNIDOC=true and runs the Jekyll build), but a user exporting SKIP_UNIDOC=true expecting the doc build to skip would be surprised.
There was a problem hiding this comment.
I've updated the doc build to accept any non-empty value for its skip flags. I've updated the PR description accordingly.
| * `SKIP_SPARK_BUILD=1`: Skip building Spark. A working Spark build is required for the docs (except the error docs) to build correctly, so set this only if you know you've already built Spark. | ||
| * `SKIP_API=1`: Skip building all the API docs. This is basically the same as setting all of the skip flags below. | ||
| * `SKIP_SCALADOC=1`: Skip the Scala and Java API docs, which includes building the Unidoc. | ||
| * `SKIP_UNIDOC=1`: Skip building the Unidoc. |
There was a problem hiding this comment.
Doc-wording nit: the SKIP_UNIDOC=1 entry ("Skip building the Unidoc") omits that the Scala/Java copy step (copy_and_update_scala_docs/copy_and_update_java_docs from target/scala-2.13/unidoc and target/javaunidoc) still runs, so the build fails there if no pre-built unidoc output exists. SKIP_SPARK_BUILD=1 carries the analogous "only set this if you've already built" caveat; SKIP_UNIDOC should too (use SKIP_SCALADOC=1 to skip the copy as well).
What changes were proposed in this pull request?
Add a couple of new
SKIP_flags to the existing doc build script:SKIP_SPARK_BUILDSKIP_UNIDOCUpdate the docs README to reflect the complete set of skip flags.
Change the docs build script to accept any non-empty value as enabling a particular skip flag.
Why are the changes needed?
#48860 added
SPARK_DOCS_IS_BUILT_ON_HOSTbut it doesn't really fit the existing "interface" of the doc build script, as clunky as it is. (The interface is basically a collection ofSKIP_flags.)This PR tweaks the new skip flags so they are more consistent with the existing ones.
The change from accepting
1only to accepting any non-empty value also makes the docs build more consistent with how skip flags are handled elsewhere, like here.Does this PR introduce any user-facing change?
No.
How was this patch tested?
I ran the doc build as follows without issue:
SKIP_UNIDOC=1 SKIP_SPARK_BUILD=1 SKIP_RDOC=1 bundle exec jekyll buildThe doc build worked as expected. The Spark build was correctly skipped, and the Scala/Java doc build was quick since it was just copying files around.
I also ran the docs with other values passed in to the skip flags and confirmed they are accepted:
Was this patch authored or co-authored using generative AI tooling?
No.