[SPARK-27439][SQL] Use analyzed plan when explaining Dataset

[viirya]

What changes were proposed in this pull request?

Because a review is resolved during analysis when we create a dataset, the content of the view is determined when the dataset is created, not when it is evaluated. Now the explain result of a dataset is not correctly consistent with the collected result of it, because we use pre-analyzed logical plan of the dataset in explain command. The explain command will analyzed the logical plan passed in. So if a view is changed after the dataset was created, the plans shown by explain command aren't the same with the plan of the dataset.

scala> spark.range(10).createOrReplaceTempView("test")
scala> spark.range(5).createOrReplaceTempView("test2")
scala> spark.sql("select * from test").createOrReplaceTempView("tmp001")
scala> val df = spark.sql("select * from tmp001")
scala> spark.sql("select * from test2").createOrReplaceTempView("tmp001")
scala> df.show
+---+
| id|
+---+
|  0|
|  1|
|  2|
|  3|
|  4|
|  5|
|  6|
|  7|
|  8|
|  9|
+---+
scala> df.explain

Before:

== Physical Plan ==
*(1) Range (0, 5, step=1, splits=12)

After:

== Physical Plan ==
*(1) Range (0, 10, step=1, splits=12)

How was this patch tested?

Manually test and unit test.

[SparkQA]

Test build #104747 has finished for PR 24415 at commit e1fe974.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[viirya]

cc @cloud-fan @dongjoon-hyun

[dongjoon-hyun]

Thank you for pinging me, @viirya . I like this PR. First of all, could you add a test case for this please? I'll take a look this more.

[viirya]

Sure. I originally thought manual test might be enough. Will add one unit test soon.

[dongjoon-hyun]

Thank you. Yes. This is important fix. I want to prevent the future regressions.

[viirya]

Added an unit test. Thanks for review.

[SparkQA]

Test build #104771 has finished for PR 24415 at commit c7b99c3.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[dongjoon-hyun]

+1, LGTM. Merged to master.
Thank you, @viirya and @HyukjinKwon.

[viirya]

Thanks!

[HyukjinKwon]

Looks good to me too.

[dongjoon-hyun]

This is reverted due to the regression. Please see the JIRA for the detail.

[viirya]

Thanks @dongjoon-hyun. I will look into how to fix it.

[HyukjinKwon]

Yes, it looks it should be reverted, and pass the logical plan as is. Thanks for quick action, @dongjoon-hyun.