[SPARK-34222][SQL][FOLLOWUP] Non-recursive implementation of buildBalancedPredicate

[gengliangwang]

What changes were proposed in this pull request?

Use a non-recursive implementation for the function buildBalancedPredicate

Why are the changes needed?

For better performance.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Existing unit tests.
Also, a quick benchmark:

  test("buildBalancedPredicate") {
    val expressions = (1 to 1000).map(_ => Literal(true))
    val start = System.currentTimeMillis()
    buildBalancedPredicate(expressions, And)
    println(System.currentTimeMillis() - start)
  }

Before: 47ms
After: 4ms

[gengliangwang]

cc @Swinky @cloud-fan @maropu

[Swinky]

for N predicates, number of recursive calls on stack are logN, (with base 2) for buildBalancedPredicate.

If we have such huge predicates then it would fail with stackoverflow in splitDisjunctivePredicates and splitConjunctivePredicates before this ?

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40288/

[SparkQA]

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40288/

[gengliangwang]

@Swinky I have updated the description. It's for better performance.

[Swinky]

Thanks @gengliangwang for this !

[SparkQA]

Test build #135706 has finished for PR 31724 at commit d0e2d74.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[gengliangwang]

@cloud-fan @maropu @Swinky Thanks for the review.
Merging to master

[HyukjinKwon]

LGTM2