Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-27463][PYTHON][FOLLOW-UP] Run the tests of Cogrouped pandas UDF #25890

Closed
wants to merge 3 commits into from

Conversation

HyukjinKwon
Copy link
Member

@HyukjinKwon HyukjinKwon commented Sep 22, 2019

What changes were proposed in this pull request?

This is a followup for #24981
Seems we mistakenly didn't added test_pandas_udf_cogrouped_map into modules.py. So we don't have official test results against that PR.

...
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf
...
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_grouped_agg
...
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_grouped_map
...
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_scalar
...
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_window
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf (21s)
...
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_grouped_map (49s)
...
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_window (58s)
...
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_scalar (82s)
...
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_grouped_agg (105s)
...

If tests fail, we should revert that PR.

Why are the changes needed?

Relevant tests should be ran.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Jenkins tests.

@HyukjinKwon
Copy link
Member Author

I am skipping Scala tests at 7d9835d I should revert 7d9835d before merging it in.

@HyukjinKwon
Copy link
Member Author

Okay, seems fortunately it's passing:

Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_cogrouped_map
...
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_cogrouped_map (39s)

@HyukjinKwon
Copy link
Member Author

I am going to merge this as soon as the tests pass to prevent further unexpected regressions in cogroup pandas UDF.

cc @BryanCutler and @ueshin

@SparkQA
Copy link

SparkQA commented Sep 22, 2019

Test build #111151 has finished for PR 25890 at commit 7d9835d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 22, 2019

Test build #111150 has finished for PR 25890 at commit 3c052e4.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 22, 2019

Test build #111152 has finished for PR 25890 at commit f6cd95a.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon
Copy link
Member Author

Merged to master.

@dongjoon-hyun
Copy link
Member

+1, late LGTM.

@ueshin
Copy link
Member

ueshin commented Sep 22, 2019

late LGTM. Thanks!

@BryanCutler
Copy link
Member

Thanks for catching this @HyukjinKwon , LGTM

@HyukjinKwon HyukjinKwon deleted the SPARK-28840 branch March 3, 2020 01:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants