Skip to content

Regenerate PySpark API documentation for Spark 3.2.1#381

Closed
xinrong-meng wants to merge 3 commits intoapache:asf-sitefrom
xinrong-meng:py_doc_3.2.1
Closed

Regenerate PySpark API documentation for Spark 3.2.1#381
xinrong-meng wants to merge 3 commits intoapache:asf-sitefrom
xinrong-meng:py_doc_3.2.1

Conversation

@xinrong-meng
Copy link
Member

@xinrong-meng xinrong-meng commented Mar 4, 2022

This PR regenerates PySpark API documentation for Spark 3.2.1.

Specifically, files under https://github.com/apache/spark-website/tree/asf-site/site/docs/3.2.1/api/python have been updated.

Currently, API of pandas API on Spark 3.2.1 are not associated with links, for example, API under https://spark.apache.org/docs/latest/api/python/reference/pyspark.pandas/series.html. So we wanted to regenerate it with correct dependencies.

How it gets updated:

  • Generated docs under spark/python/doc against v3.2.1 Spark
  • Copy generated files from spark/python/docs/build/html to spark-website/site/docs/3.2.1/api/python.

@srowen
Copy link
Member

srowen commented Mar 4, 2022

Out of curiosity what creates all the changes - is it just formatting or any substantive change?

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is wrong, isn't it? @xinrong-databricks

- <title>abc &#8212; PySpark 3.2.1 documentation</title>
+ <title>abc &#8212; PySpark master documentation</title>

cc @HyukjinKwon and @gengliangwang

Copy link
Member

@HyukjinKwon HyukjinKwon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should regenerate the docs by setting GIT_HASH and RELEASE_VERSION environments properly (see https://github.com/apache/spark/blob/master/python/docs/source/conf.py). BTW, I wonder why this failed to generate the docs this time .. I fixed it here apache/spark@03750c0

@xinrong-meng xinrong-meng changed the title Regenerate PySpark documentation for Spark 3.2.1 [WIP] Regenerate PySpark documentation for Spark 3.2.1 Mar 7, 2022
@gengliangwang
Copy link
Member

Out of curiosity what creates all the changes - is it just formatting or any substantive change?

Same question here.
As many committers are watching this repo closely, @xinrong-databricks it would be great if you provide a summary of the changes in the PR description.

@xinrong-meng xinrong-meng changed the title [WIP] Regenerate PySpark documentation for Spark 3.2.1 Regenerate PySpark documentation for Spark 3.2.1 Mar 8, 2022
@xinrong-meng xinrong-meng changed the title Regenerate PySpark documentation for Spark 3.2.1 Regenerate PySpark API documentation for Spark 3.2.1 Mar 8, 2022
@xinrong-meng
Copy link
Member Author

Sorry for the confusion. I added PR descriptions. Let me know if there are any questions.

FYI @HyukjinKwon @gengliangwang @srowen @dongjoon-hyun

@srowen
Copy link
Member

srowen commented Mar 8, 2022

It's hard to evaluate by looking at the diff but if it was merely regenerated from the 3.2.1 release to fix rendering of links, that seems OK

@HyukjinKwon
Copy link
Member

Merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants