Skip to content

[SPARK-46208][PS][DOCS] Adding a link for latest Pandas API specifications.#44115

Closed
itholic wants to merge 2 commits into
apache:masterfrom
itholic:use_exact_pandas_version
Closed

[SPARK-46208][PS][DOCS] Adding a link for latest Pandas API specifications.#44115
itholic wants to merge 2 commits into
apache:masterfrom
itholic:use_exact_pandas_version

Conversation

@itholic
Copy link
Copy Markdown
Contributor

@itholic itholic commented Dec 2, 2023

What changes were proposed in this pull request?

This PR proposes to adding a link for latest Pandas API specifications.

Why are the changes needed?

To increase the usability of documents

Does this PR introduce any user-facing change?

No.

How was this patch tested?

The existing CI should pass

Was this patch authored or co-authored using generative AI tooling?

No.

@itholic
Copy link
Copy Markdown
Contributor Author

itholic commented Dec 2, 2023

cc @HyukjinKwon @zhengruifeng

This page gives an overview of all public pandas API on Spark.

.. note::
pandas API on Spark follows the API specifications of latest pandas release.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we cannot always support the latest pandas version in some cases, especially when they introduce the breaking changes.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should just keep it as is for now. It's a bit of overhead to keep changing this monthly

Copy link
Copy Markdown
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that the monthly Pandas release cycle, it's also impossible to update this documentation at every month.

Screenshot 2023-12-02 at 1 17 43 PM

@itholic
Copy link
Copy Markdown
Contributor Author

itholic commented Dec 3, 2023

Sounds reasonable to me. Let me just close this ticket.

@itholic itholic closed this Dec 3, 2023
@itholic itholic reopened this Dec 3, 2023
Comment thread python/docs/source/reference/pyspark.pandas/index.rst Outdated
@itholic itholic changed the title [SPARK-46208][PS][DOCS] Use specific Pandas version for API specifications. [SPARK-46208][PS][DOCS] Adding a link for latest Pandas API specifications. Dec 3, 2023
Co-authored-by: Haejoon Lee <44108233+itholic@users.noreply.github.com>
@HyukjinKwon
Copy link
Copy Markdown
Member

Merged to master.

asl3 pushed a commit to asl3/spark that referenced this pull request Dec 5, 2023
…tions

### What changes were proposed in this pull request?

This PR proposes to adding a link for latest Pandas API specifications.

### Why are the changes needed?

To increase the usability of documents

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

The existing CI should pass

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#44115 from itholic/use_exact_pandas_version.

Lead-authored-by: Haejoon Lee <haejoon.lee@databricks.com>
Co-authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
dbatomic pushed a commit to dbatomic/spark that referenced this pull request Dec 11, 2023
…tions

### What changes were proposed in this pull request?

This PR proposes to adding a link for latest Pandas API specifications.

### Why are the changes needed?

To increase the usability of documents

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

The existing CI should pass

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#44115 from itholic/use_exact_pandas_version.

Lead-authored-by: Haejoon Lee <haejoon.lee@databricks.com>
Co-authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
@itholic itholic deleted the use_exact_pandas_version branch January 1, 2024 23:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants