Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-33171][INFRA] Mark ParquetV*FilterSuite/ParquetV*SchemaPruningSuite as ExtendedSQLTest #30068

Closed
wants to merge 2 commits into from
Closed

[SPARK-33171][INFRA] Mark ParquetV*FilterSuite/ParquetV*SchemaPruningSuite as ExtendedSQLTest #30068

wants to merge 2 commits into from

Conversation

dongjoon-hyun
Copy link
Member

@dongjoon-hyun dongjoon-hyun commented Oct 16, 2020 •
edited
Loading

What changes were proposed in this pull request?

This PR aims to mark ParquetV1FilterSuite and ParquetV2FilterSuite as ExtendedSQLTest.

  • ParquetV1FilterSuite/ParquetV2FilterSuite
  • ParquetV1SchemaPruningSuite/ParquetV2SchemaPruningSuite

Why are the changes needed?

Currently, sql - other tests is the longest job. This PR will move the above tests to sql - slow tests job.

BEFORE

AFTER

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Pass the Github Action with the reduced time.

@dongjoon-hyun dongjoon-hyun marked this pull request as draft October 16, 2020 16:07
@dongjoon-hyun dongjoon-hyun mentioned this pull request Oct 16, 2020
Closed
5 tasks
@SparkQA
Copy link

SparkQA commented Oct 16, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34508/

@dongjoon-hyun dongjoon-hyun changed the title [SPARK-XXX][INFRA] Mark Parquet*FilterSuite as ExtendedSQLTest [SPARK-33171][INFRA] Mark Parquet*FilterSuite as ExtendedSQLTest Oct 16, 2020
@SparkQA
Copy link

SparkQA commented Oct 16, 2020

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34508/

@dongjoon-hyun dongjoon-hyun changed the title [SPARK-33171][INFRA] Mark Parquet*FilterSuite as ExtendedSQLTest [SPARK-33171][INFRA] Mark ParquetV*FilterSuite/ParquetV*SchemaPruningSuite as ExtendedSQLTest Oct 16, 2020
@SparkQA
Copy link

SparkQA commented Oct 16, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34514/

@SparkQA
Copy link

SparkQA commented Oct 16, 2020

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34514/

@dongjoon-hyun dongjoon-hyun marked this pull request as ready for review October 16, 2020 19:33
@dongjoon-hyun
Copy link
Member Author

Could you review this, @srowen , @gengliangwang , @HyukjinKwon , @viirya ?

viirya
viirya approved these changes Oct 16, 2020
View reviewed changes
Copy link
Member

@viirya viirya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks ok. Maybe we can rename sql - slow tests? Are ParquetVFilterSuite/ParquetVSchemaPruningSuite slow tests in particular?

@dongjoon-hyun
Copy link
Member Author

Ya. We can. Currently, it seems that @HyukjinKwon designed this for easy consistent categorization in Hive/SQL.
Screen Shot 2020-10-16 at 12 50 36 PM

@dongjoon-hyun
Copy link
Member Author

For naming things, we can switch it with a follow-up as a easy fix without CI passing. So, I'll merge this first to save our community time. Thank you so much, @viirya !

@dongjoon-hyun dongjoon-hyun deleted the MOVE3 branch October 16, 2020 19:53
@dongjoon-hyun
Copy link
Member Author

Merged to master

dongjoon-hyun added a commit that referenced this pull request Oct 16, 2020
@dongjoon-hyun
[SPARK-33171][INFRA] Mark ParquetV*FilterSuite/ParquetV*SchemaPruning…
b66bd79 
…Suite as ExtendedSQLTest

### What changes were proposed in this pull request?

This PR aims to mark ParquetV1FilterSuite and ParquetV2FilterSuite as `ExtendedSQLTest`.
- ParquetV1FilterSuite/ParquetV2FilterSuite
- ParquetV1SchemaPruningSuite/ParquetV2SchemaPruningSuite

### Why are the changes needed?

Currently, `sql - other tests` is the longest job. This PR will move the above tests to `sql - slow tests` job.

**BEFORE**
- https://github.com/apache/spark/runs/1264150802 (1 hour 37 minutes)

**AFTER**
- https://github.com/apache/spark/pull/30068/checks?check_run_id=1265879896 (1 hour 21 minutes)

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Pass the Github Action with the reduced time.

Closes #30068 from dongjoon-hyun/MOVE3.

Lead-authored-by: Dongjoon Hyun <dongjoon@apache.org>
Co-authored-by: Dongjoon Hyun <dhyun@apple.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
@dongjoon-hyun
Copy link
Member Author

Also, cherry-picked to branch-3.0 to reduce the waiting time there.

@SparkQA
Copy link

SparkQA commented Oct 16, 2020

Test build #129902 has finished for PR 30068 at commit eadc235.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Oct 16, 2020

Test build #129908 has finished for PR 30068 at commit 767763b.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon
Copy link
Member

Nice!

holdenk pushed a commit to holdenk/spark that referenced this pull request Oct 27, 2020
@dongjoon-hyun
[SPARK-33171][INFRA] Mark ParquetV*FilterSuite/ParquetV*SchemaPruning…
ff699bd 
…Suite as ExtendedSQLTest

### What changes were proposed in this pull request?

This PR aims to mark ParquetV1FilterSuite and ParquetV2FilterSuite as `ExtendedSQLTest`.
- ParquetV1FilterSuite/ParquetV2FilterSuite
- ParquetV1SchemaPruningSuite/ParquetV2SchemaPruningSuite

### Why are the changes needed?

Currently, `sql - other tests` is the longest job. This PR will move the above tests to `sql - slow tests` job.

**BEFORE**
- https://github.com/apache/spark/runs/1264150802 (1 hour 37 minutes)

**AFTER**
- https://github.com/apache/spark/pull/30068/checks?check_run_id=1265879896 (1 hour 21 minutes)

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Pass the Github Action with the reduced time.

Closes apache#30068 from dongjoon-hyun/MOVE3.

Lead-authored-by: Dongjoon Hyun <dongjoon@apache.org>
Co-authored-by: Dongjoon Hyun <dhyun@apple.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@viirya viirya viirya approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@dongjoon-hyun @SparkQA @HyukjinKwon @viirya