Skip to content

[SPARK-33431][CORE][SQL][DSTREAM] Use Option.contains instead of instanceOfOption == Some(x)#30353

Closed
LuciferYang wants to merge 2 commits intoapache:masterfrom
LuciferYang:SPARK-33431
Closed

[SPARK-33431][CORE][SQL][DSTREAM] Use Option.contains instead of instanceOfOption == Some(x)#30353
LuciferYang wants to merge 2 commits intoapache:masterfrom
LuciferYang:SPARK-33431

Conversation

@LuciferYang
Copy link
Contributor

@LuciferYang LuciferYang commented Nov 12, 2020

What changes were proposed in this pull request?

There are some code like InstanceOfOption == Some(x), for this comparison, we need to wrap x as another Option instance. In this pr use the Option.contains method to simplify the above way and avoid to create new Option instance.

Before:

option == Some(x)
Some(x) != option

After:

option.contains(x)
!option.contains(x)

Why are the changes needed?

Simplify code and reduce object creation.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Pass the Jenkins or GitHub Action

@maropu
Copy link
Member

maropu commented Nov 12, 2020

Before:
option == Some(x)
Some(x) != option
After:
option == Some(x)
Some(x) != option

Seems the before/after examples in the description are the same?

Anyway, the changes sound reasonable. The other sub-modules (e.g., mllib) do not have the pattern (option == Some(x))?

@maropu maropu changed the title [SPARK-33431][CORE][SQL]Use Option.contains instead of instanceOfOption == Some(x) [SPARK-33431][CORE][SQL] Use Option.contains instead of instanceOfOption == Some(x) Nov 12, 2020
// If no attempt is specified, or there is no attemptId for attempts, return all attempts
attemptId
.map { id => app.attempts.filter(_.info.attemptId == Some(id)) }
.map { id => app.attempts.filter(_.info.attemptId.contains(id)) }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, these all look like a performance non-sensitive code path. Is it worthwhile to sweep and fix? I am not sure because it will make more conflicts when reverting, backporting, etc. Option == Some(id) can be sort of more readable in a way.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mainly to reduce unnecessary object creation, if the readability is affected, it can be left unchanged

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have the same comment; I don't think it matters much either way. If it's just 14 instances, OK.

@SparkQA
Copy link

SparkQA commented Nov 12, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35599/

@LuciferYang
Copy link
Contributor Author

LuciferYang commented Nov 12, 2020

@maropu

Seems the before/after examples in the description are the same?

The description has been corrected

Anyway, the changes sound reasonable. The other sub-modules (e.g., mllib) do not have the pattern (option == Some(x))?

Other modules also have, I can further append new commits to this pr if necessary

@SparkQA
Copy link

SparkQA commented Nov 12, 2020

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35599/

@SparkQA
Copy link

SparkQA commented Nov 12, 2020

Test build #130993 has finished for PR 30353 at commit 726781f.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@LuciferYang LuciferYang marked this pull request as draft November 13, 2020 10:56
@LuciferYang LuciferYang changed the title [SPARK-33431][CORE][SQL] Use Option.contains instead of instanceOfOption == Some(x) [WIP][SPARK-33431][CORE][SQL] Use Option.contains instead of instanceOfOption == Some(x) Nov 13, 2020
@LuciferYang LuciferYang changed the title [WIP][SPARK-33431][CORE][SQL] Use Option.contains instead of instanceOfOption == Some(x) [WIP][SPARK-33431][CORE][SQL][DSTREAM] Use Option.contains instead of instanceOfOption == Some(x) Nov 13, 2020
@LuciferYang
Copy link
Contributor Author

Anyway, the changes sound reasonable. The other sub-modules (e.g., mllib) do not have the pattern (option == Some(x))?

@maropu After Address e95295a, other related issues all in the test code

@LuciferYang LuciferYang changed the title [WIP][SPARK-33431][CORE][SQL][DSTREAM] Use Option.contains instead of instanceOfOption == Some(x) [SPARK-33431][CORE][SQL][DSTREAM] Use Option.contains instead of instanceOfOption == Some(x) Nov 13, 2020
@LuciferYang LuciferYang marked this pull request as ready for review November 13, 2020 11:51
@SparkQA
Copy link

SparkQA commented Nov 13, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35666/

@SparkQA
Copy link

SparkQA commented Nov 13, 2020

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35666/

@SparkQA
Copy link

SparkQA commented Nov 13, 2020

Test build #131062 has finished for PR 30353 at commit e95295a.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@LuciferYang LuciferYang deleted the SPARK-33431 branch June 6, 2022 03:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants