[SPARK-40677][CONNECT][BUILD] Shade more dependency to be able to run separately

[HyukjinKwon]

What changes were proposed in this pull request?

This PR makes the following dependencies shaded:

com.google.android:annotations
com.google.api.grpc:proto-google-common-proto
io.perfmark:perfmark-api
org.codehaus.mojo:animal-sniffer-annotations
com.google.errorprone:error_prone_annotations
com.google.j2objc:j2objc-annotations

Before #38109, it worked because related dependences pulled together but now we don't as Spark Connect would be a single jar. This issue has existed from the very first place.

Why are the changes needed?

Otherwise, the tests fails if you build Spark Connect with Maven. SBT does not have the issue because it does the assemply with all dependencies.

Does this PR introduce any user-facing change?

No, the codes are not released out yet.

How was this patch tested?

Manually tested via Maven:

./build/mvn clean package
./python/run-tests --testnames 'pyspark.sql.tests.test_connect_basic'

[HyukjinKwon]

cc @LuciferYang FYI

[HyukjinKwon]

cc @amaliujia too FYI

[HyukjinKwon]

Build: https://github.com/HyukjinKwon/spark/actions/runs/3197581888

[amaliujia]

question:

how do you decide which package needs a shading?

[HyukjinKwon]

Oh, these were jars missing, andI manually added them (see also #38109 (comment))

[HyukjinKwon]

Probably there should be a better way I believe but I am going to merge it first to fix the broken test cases with Maven.

[HyukjinKwon]

Merged to master.

[LuciferYang]

@HyukjinKwon Sorry for the late reply

The relocation rule may be incorrect

I think it should be

<pattern>android.annotation</pattern>
<shadedPattern>${spark.shade.packageName}.connect.android.annotation</shadedPattern>

and unzip the assembly jar, the contents are as follows:

ls

META-INF	android		com		google		grpc		org		spark

Not all contents are placed in ${spark.shade.packageName}.connect, which may require further check

[HyukjinKwon]

Hey sure, please go ahead with a followup. I just rushed to merge to fix up the broken tests with Maven :-).

[LuciferYang]

OK~ Maybe do this tomorrow, a little tired today