-
Notifications
You must be signed in to change notification settings - Fork 28k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-15734][SQL] Avoids printing internal row in explain output #13471
Conversation
8ec1684
to
2e679f7
Compare
cc @cloud-fan |
LGTM |
override protected def stringArgs = Iterator(output) | ||
override protected def stringArgs: Iterator[Any] = { | ||
if (data.isEmpty) { | ||
Iterator("Empty", output) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe "EmptyRelation"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just to make sure, do you prefer the following output?
LocalRelation EmptyRelation, [_1#58,_2#59,_3#60]
Instead of
LocalRelation Empty, [_1#58,_2#59,_3#60]
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point... How about just ""? Both "Empty" and "EmptyRelation" look like some class name, while "" reads more like an annotation. (This is pretty subjective though.)
LGTM except for minor naming issue. |
Test build #59867 has finished for PR 13471 at commit
|
Test build #59874 has finished for PR 13471 at commit
|
Merging to master and branch-2.0. |
## What changes were proposed in this pull request? This PR avoids printing internal rows in explain output for some operators. **Before change:** ``` scala> (1 to 10).toSeq.map(_ => (1,2,3)).toDF().createTempView("df3") scala> spark.sql("select * from df3 where 1=2").explain(true) ... == Analyzed Logical Plan == _1: int, _2: int, _3: int Project [_1#37,_2#38,_3#39] +- Filter (1 = 2) +- SubqueryAlias df3 +- LocalRelation [_1#37,_2#38,_3#39], [[0,1,2,3],[0,1,2,3],[0,1,2,3],[0,1,2,3],[0,1,2,3],[0,1,2,3],[0,1,2,3],[0,1,2,3],[0,1,2,3],[0,1,2,3]] ... == Physical Plan == LocalTableScan [_1#37,_2#38,_3#39] ``` **After change:** ``` scala> spark.sql("select * from df3 where 1=2").explain(true) ... == Analyzed Logical Plan == _1: int, _2: int, _3: int Project [_1#58,_2#59,_3#60] +- Filter (1 = 2) +- SubqueryAlias df3 +- LocalRelation [_1#58,_2#59,_3#60] ... == Physical Plan == LocalTableScan <empty>, [_1#58,_2#59,_3#60] ``` ## How was this patch tested? Manual test. Author: Sean Zhong <seanzhong@databricks.com> Closes #13471 from clockfly/verbose_breakdown_5. (cherry picked from commit 985d532) Signed-off-by: Cheng Lian <lian@databricks.com>
What changes were proposed in this pull request?
This PR avoids printing internal rows in explain output for some operators.
Before change:
After change:
How was this patch tested?
Manual test.