[WIP][SPARK-35137][SQL] Revise outputpartitioning in some SparkPlan

[ulysses-you]

What changes were proposed in this pull request?

Revise the outputPartitioning number in some SparkPlan

• UnionExec
• RowDataSourceScanExec
• LogicalRDD
• RDDScanExec
• ExternalRDDScanExec
• LocalTableScanExec
• ExpandExec
• BroadcastNestedLoopJoinExec
• CoGroupExec
• DataSourceV2ScanExecBase
• CustomShuffleReaderExec

The mian change follow the logical

1. if rdd is known, using rdd.partitions.length
2. using child.outputPartitioning.numPartitions
3. check SinglePartition

Why are the changes needed?

Make outputPartitioning more reasonable.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Pass CI.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42161/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42161/

[SparkQA]

Test build #137630 has finished for PR 32236 at commit 6efe5fa.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42385/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42385/

[SparkQA]

Test build #137855 has finished for PR 32236 at commit 4fd5111.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[ulysses-you]

Seq(1 -> "a").toDF("i", "j") output partitioning number is 1 so it's SinglePartition.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42424/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42424/

[SparkQA]

Test build #137899 has finished for PR 32236 at commit e104a30.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42589/

[SparkQA]

Test build #138777 has finished for PR 32236 at commit a33157b.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43301/

[github-actions]

We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!