[SPARK-28716][SQL] Add id to Exchange and Subquery's stringArgs method for easier identifying their reuses in query plans #25434
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
ok to test |
dongjoon-hyun
changed the title
|
ok to test |
|
Test build #109045 has finished for PR 25434 at commit
|
|
Test build #109067 has finished for PR 25434 at commit
|
|
Can you put explain output differences with/without this pr in the PR description? Also, It'd be better to put the simple query to reproduce the case. |
maropu
reviewed
Aug 15, 2019
| assert(output.contains( | ||
| """== BroadcastExchange HashedRelationBroadcastMode(List(input[0, bigint, false])) == | ||
| assert(output.replaceAll("\\[id=#\\d+\\]", "[id=#x]").contains( | ||
| """== BroadcastExchange HashedRelationBroadcastMode(List(input[0, bigint, false])), [id=#x] == |
There was a problem hiding this comment.
We need to always print this id? only for reuse cases?
There was a problem hiding this comment.
Then you need to make two passes over the tree. The first one to figure out which exchanges are reused and the second to print the tree. This seems a bit excessive for a debugging tool.
|
Merging to master. |
rshkv
pushed a commit
to palantir/spark
that referenced
this pull request
Jan 29, 2021
…d for easier identifying their reuses in query plans Add id to Exchange and Subquery's stringArgs method for easier identifying their reuses in query plans, for example: ``` ReusedExchange d_date_sk#827, BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint))) [id=apache#2710] ``` Where `2710` is the id of the reused exchange. Passes existing tests Closes apache#25434 from dbaliafroozeh/ImplementStringArgsExchangeSubqueryExec. Authored-by: Ali Afroozeh <ali.afroozeh@databricks.com> Signed-off-by: herman <herman@databricks.com>
rshkv
pushed a commit
to palantir/spark
that referenced
this pull request
Jan 29, 2021
…d for easier identifying their reuses in query plans Add id to Exchange and Subquery's stringArgs method for easier identifying their reuses in query plans, for example: ``` ReusedExchange d_date_sk#827, BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint))) [id=apache#2710] ``` Where `2710` is the id of the reused exchange. Passes existing tests Closes apache#25434 from dbaliafroozeh/ImplementStringArgsExchangeSubqueryExec. Authored-by: Ali Afroozeh <ali.afroozeh@databricks.com> Signed-off-by: herman <herman@databricks.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Add id to Exchange and Subquery's stringArgs method for easier identifying their reuses in query plans, for example:
Where
2710is the id of the reused exchange.How was this patch tested?
Passes existing tests