Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-42279][PS][TESTS] Simplify pyspark.pandas.tests.test_resample #39847

Closed

Conversation

zhengruifeng
Copy link
Contributor

What changes were proposed in this pull request?

Simplify test_resample

Why are the changes needed?

test_resample may take more than 20min, this PR remove redundant combinations.

In my local env, 249 sec -> 30 sec.

Does this PR introduce any user-facing change?

No, test only

How was this patch tested?

updated tests

@zhengruifeng zhengruifeng changed the title [SPARK-42279][PS][TESTS] Simplify test_resample [SPARK-42279][PS][TESTS] Simplify pyspark.pandas.tests.test_resample Feb 1, 2023
@zhengruifeng
Copy link
Contributor Author

master: https://github.com/apache/spark/actions/runs/4063229672/jobs/6995258363

Starting test(python3.9): pyspark.pandas.tests.test_resample (temp output: /__w/spark/spark/python/target/100da025-d91f-49a5-a584-c5b6b73b5fe8/python3.9__pyspark.pandas.tests.test_resample__eaa8shha.log)
Finished test(python3.9): pyspark.pandas.tests.test_resample (715s)

in this PR

Starting test(python3.9): pyspark.pandas.tests.test_resample (temp output: /__w/spark/spark/python/target/ee3dab52-733a-4fd0-8df8-94753436bc56/python3.9__pyspark.pandas.tests.test_resample__5jmdqzv1.log)
Finished test(python3.9): pyspark.pandas.tests.test_resample (58s)

@zhengruifeng
Copy link
Contributor Author

cc @HyukjinKwon

@HyukjinKwon
Copy link
Member

Merged to master and branch-3.4.

HyukjinKwon pushed a commit that referenced this pull request Feb 2, 2023
### What changes were proposed in this pull request?
Simplify `test_resample`

### Why are the changes needed?
`test_resample` may take more than 20min, this PR remove redundant combinations.

In my local env,  249 sec -> 30 sec.

### Does this PR introduce _any_ user-facing change?
No, test only

### How was this patch tested?
updated tests

Closes #39847 from zhengruifeng/pandas_opt_test_resample.

Authored-by: Ruifeng Zheng <ruifengz@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
(cherry picked from commit 9043224)
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
@zhengruifeng zhengruifeng deleted the pandas_opt_test_resample branch February 2, 2023 02:47
snmvaughan pushed a commit to snmvaughan/spark that referenced this pull request Jun 20, 2023
### What changes were proposed in this pull request?
Simplify `test_resample`

### Why are the changes needed?
`test_resample` may take more than 20min, this PR remove redundant combinations.

In my local env,  249 sec -> 30 sec.

### Does this PR introduce _any_ user-facing change?
No, test only

### How was this patch tested?
updated tests

Closes apache#39847 from zhengruifeng/pandas_opt_test_resample.

Authored-by: Ruifeng Zheng <ruifengz@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
(cherry picked from commit 9043224)
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants