Skip to content

[SPARK-49370][BUILD] maven.scaladoc.skip should not affect test code compilation#47858

Closed
pan3793 wants to merge 1 commit intoapache:masterfrom
pan3793:SPARK-49370
Closed

[SPARK-49370][BUILD] maven.scaladoc.skip should not affect test code compilation#47858
pan3793 wants to merge 1 commit intoapache:masterfrom
pan3793:SPARK-49370

Conversation

@pan3793
Copy link
Member

@pan3793 pan3793 commented Aug 23, 2024

What changes were proposed in this pull request?

Move maven.scaladoc.skip from the scala-maven-plugin's global configuration to its attach-scaladocs execution's configuration.

Why are the changes needed?

maven.scaladoc.skip was added in SPARK-43461 to speed up the dev/make-distribution.sh by skipping scaladocs generation, but it was wrongly applied to a broader scope.

Does this PR introduce any user-facing change?

It affects developers who use build/mvn deploy -Dmaven.scaladoc.skip=true to deploy the artifacts case, without this patch, wrong <module>-tests.jars (don't contain the test classes but only resources) will be produced.

How was this patch tested?

build/mvn clean package -DskipTests -Dmaven.scaladoc.skip=true -pl core -am

before

$ ll core/target/spark-core_2.13-4.0.0-SNAPSHOT-tests.jar
-rw-r--r--@ 1 chengpan  staff   381K Aug 23 23:55 core/target/spark-core_2.13-4.0.0-SNAPSHOT-tests.jar

$ jar tf core/target/spark-core_2.13-4.0.0-SNAPSHOT-tests.jar | grep class | head
<nothing>

after

$ ll core/target/spark-core_2.13-4.0.0-SNAPSHOT-tests.jar
-rw-r--r--@ 1 chengpan  staff   5.7M Aug 24 00:12 core/target/spark-core_2.13-4.0.0-SNAPSHOT-tests.jar

$ jar tf core/target/spark-core_2.13-4.0.0-SNAPSHOT-tests.jar | grep class | head
test/org/apache/spark/JavaTaskContextCompileCheck.class
test/org/apache/spark/Java8RDDAPISuite.class
test/org/apache/spark/JavaAPISuite$SomeCustomClass.class
test/org/apache/spark/JavaAPISuite$BuggyMapFunction.class
test/org/apache/spark/JavaAPISuite$DoubleComparator.class
test/org/apache/spark/JavaAPISuite.class
test/org/apache/spark/JavaTaskContextCompileCheck$JavaTaskCompletionListenerImpl.class
test/org/apache/spark/JavaTaskContextCompileCheck$JavaTaskFailureListenerImpl.class
test/org/apache/spark/JavaAPISuite$Class2.class
test/org/apache/spark/JavaAPISuite$1.class

Was this patch authored or co-authored using generative AI tooling?

No.

@github-actions github-actions bot added the BUILD label Aug 23, 2024
@pan3793
Copy link
Member Author

pan3793 commented Aug 25, 2024

cc @wangyum @LuciferYang

@HyukjinKwon
Copy link
Member

Merged to master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants