Skip to content

Conversation

@xinrong-meng
Copy link
Member

@xinrong-meng xinrong-meng commented Jun 3, 2021

What changes were proposed in this pull request?

Adjust the check_exact parameter for non-numeric columns to ensure pandas-on-Spark tests passed with all pandas versions.

Why are the changes needed?

pd.testing utils are utilized in pandas-on-Spark tests.
Due to pandas-dev/pandas#35446, check_exact=True for non-numeric columns doesn't work for older pd.testing utils, e.g. assert_series_equal. We wanted to adjust that to ensure pandas-on-Spark tests pass for all pandas versions.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Existing unit tests.

Keyword: SPARK-35337

@SparkQA
Copy link

SparkQA commented Jun 3, 2021

Test build #139298 has finished for PR 32772 at commit 787b3ac.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jun 3, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43821/

@SparkQA
Copy link

SparkQA commented Jun 3, 2021

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43821/

@xinrong-meng xinrong-meng changed the title [WIP][SPARK-35599][PYTHON] Adjust check_exact parameter for older pd.testing [SPARK-35599][PYTHON] Adjust check_exact parameter for older pd.testing Jun 4, 2021
@xinrong-meng xinrong-meng marked this pull request as ready for review June 4, 2021 18:30
@SparkQA
Copy link

SparkQA commented Jun 5, 2021

Test build #139358 has finished for PR 32772 at commit d3f44f0.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jun 5, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43880/

@SparkQA
Copy link

SparkQA commented Jun 5, 2021

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43880/

Copy link
Member

@ueshin ueshin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@xinrong-meng
Copy link
Member Author

Doc builds failed. I will rebase and retest.

@SparkQA
Copy link

SparkQA commented Jun 5, 2021

Test build #139362 has finished for PR 32772 at commit 59ad0a7.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jun 5, 2021

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43884/

@HyukjinKwon
Copy link
Member

Merged to master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants