Skip to content

Conversation

@HyukjinKwon
Copy link
Member

What changes were proposed in this pull request?

This PR documents DataFrame.mapInArrow in PySpark (#34505).
I initially planned to document like DataFrame.mapInPandas (#25025) but decided to don't publicly document like that because this is an unstable developer API.

Why are the changes needed?

For users to refer to, and use.

Does this PR introduce any user-facing change?

Yes, it documents DataFrame.mapInArrow as below:

Screen Shot 2021-11-17 at 1 27 18 PM

How was this patch tested?

Manually tested by building PySpark documentation.

@SparkQA
Copy link

SparkQA commented Nov 17, 2021

Test build #145313 has finished for PR 34624 at commit 79ec116.

  • This patch fails to build.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon
Copy link
Member Author

retest this please

@SparkQA
Copy link

SparkQA commented Nov 17, 2021

Test build #145314 has finished for PR 34624 at commit 79ec116.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Nov 17, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49784/

@SparkQA
Copy link

SparkQA commented Nov 17, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49785/

@SparkQA
Copy link

SparkQA commented Nov 17, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49784/

@SparkQA
Copy link

SparkQA commented Nov 17, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49785/

@HyukjinKwon
Copy link
Member Author

cc @viirya mind taking a quick look please?

@viirya
Copy link
Member

viirya commented Nov 18, 2021

lgtm

@HyukjinKwon
Copy link
Member Author

Thanks! Merged to master.

@HyukjinKwon
Copy link
Member Author

Merged to master.

@HyukjinKwon HyukjinKwon deleted the SPARK-37230 branch January 4, 2022 00:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants