[MINOR][DOCS] Fix some links for python api doc

[yaooqinn]

What changes were proposed in this pull request?

the link for partition discovery is malformed, because for releases, there will contains /docs/<version>/ in the full URL.

Why are the changes needed?

fix doc

Does this PR introduce any user-facing change?

no

How was this patch tested?

SKIP_SCALADOC=1 SKIP_RDOC=1 SKIP_SQLDOC=1 jekyll serve locally verified

[yaooqinn]

cc @HyukjinKwon

[SparkQA]

Test build #120327 has finished for PR 28017 at commit f6bd4ec.

• This patch fails Python style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[HyukjinKwon]

@yaooqinn you can just use the http://... links.

Seems the relative links work in your local build but doesn't work when it's released because the documentation root is rebased.

[yaooqinn]

thanks,fixed

[SparkQA]

Test build #120331 has finished for PR 28017 at commit 2fd8b3b.

• This patch fails PySpark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #120339 has finished for PR 28017 at commit b4d0fc7.

• This patch fails Python style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #120344 has finished for PR 28017 at commit 5346870.

• This patch fails to generate documentation.
• This patch merges cleanly.
• This patch adds no public classes.

[yaooqinn]

retest this please

[SparkQA]

Test build #120347 has finished for PR 28017 at commit 5346870.

• This patch fails to generate documentation.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #120348 has finished for PR 28017 at commit 5346870.

• This patch fails to generate documentation.
• This patch merges cleanly.
• This patch adds no public classes.

[HyukjinKwon]

retest this please

[SparkQA]

Test build #120350 has finished for PR 28017 at commit 5346870.

• This patch fails to generate documentation.
• This patch merges cleanly.
• This patch adds no public classes.

[HyukjinKwon]

Hm?

[HyukjinKwon]

Seems like different avro is being picked up during unidoc for some reasons:

[info] Main Java API documentation to /home/jenkins/workspace/SparkPullRequestBuilder@6/target/javaunidoc...
[error] /home/jenkins/workspace/SparkPullRequestBuilder@6/core/src/main/scala/org/apache/spark/serializer/GenericAvroSerializer.scala:123: value createDatumWriter is not a member of org.apache.avro.generic.GenericData
[error]     writerCache.getOrElseUpdate(schema, GenericData.get.createDatumWriter(schema))
[error]                                                         ^
[info] No documentation generated with unsuccessful compiler run
[error] one error found

I faced a long ago. I had to just skip it (#17669) when it's built against hadoop-2.6 profile because explicitly setting dependencies didn't work (#17642, #17651)

[HyukjinKwon]

Can you try the below?

diff --git a/project/SparkBuild.scala b/project/SparkBuild.scala
index 4e1badb5e90..60319d81e16 100644
--- a/project/SparkBuild.scala
+++ b/project/SparkBuild.scala
@@ -625,7 +625,8 @@ object DependencyOverrides {
     dependencyOverrides += "com.google.guava" % "guava" % "14.0.1",
     dependencyOverrides += "commons-io" % "commons-io" % "2.4",
     dependencyOverrides += "xerces" % "xercesImpl" % "2.12.0",
-    dependencyOverrides += "jline" % "jline" % "2.14.6")
+    dependencyOverrides += "jline" % "jline" % "2.14.6",
+    dependencyOverrides += "org.apache.avro" % "avro" % "1.8.2")
 }

 /**

[HyukjinKwon]

@yaooqinn, can you make a separate PR to pin the avro version specifically in SBT? seems it fixes the issue. FYI @HeartSaVioR and @cloud-fan from the mailing list.

[HyukjinKwon]

Oops, mistake.

[SparkQA]

Test build #120359 has finished for PR 28017 at commit aaa1594.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[yaooqinn]

retest this please

[SparkQA]

Test build #120384 has finished for PR 28017 at commit a1fbefe.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #120385 has finished for PR 28017 at commit a1fbefe.

• This patch fails PySpark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[HyukjinKwon]

retest this please

[SparkQA]

Test build #120388 has finished for PR 28017 at commit a1fbefe.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[HyukjinKwon]

Merged to master and branch-3.0.