Skip to content

[SPARK-27518][SQL][Web UI] Show statistics in Optimized Logical Plan in the "Details" On SparkSQL ui page when CBO is enabled#24413

Closed
pengbo wants to merge 4 commits intoapache:masterfrom
pengbo:show_stats_in_detail
Closed

[SPARK-27518][SQL][Web UI] Show statistics in Optimized Logical Plan in the "Details" On SparkSQL ui page when CBO is enabled#24413
pengbo wants to merge 4 commits intoapache:masterfrom
pengbo:show_stats_in_detail

Conversation

@pengbo
Copy link
Contributor

@pengbo pengbo commented Apr 19, 2019

What changes were proposed in this pull request?

Statistics snapshot info with current query is really helpful to find out why the query runs slowly, especially caused by bad rejoin order when CBO is enabled. This PR is to show statistics in Optimized Logical Plan in the "Details" On SparkSQL ui page when CBO is enabled.

Main changes:

  1. Refactoring QueryExecution String method to avoid duplicate code
  2. Add statistic info to physicalPlanDescription when CBO is enabled

SPARK-27518-1

How was this patch tested?

Manual test & according UT has been added

@AmplabJenkins
Copy link

Can one of the admins verify this patch?

@pengbo
Copy link
Contributor Author

pengbo commented Apr 23, 2019

cc @cloud-fan @srowen

@pengbo pengbo changed the title [SPARK-27518][SQL][UI] Show statistics in Optimized Logical Plan in the "Details" On SparkSQL ui page when CBO is enabled [SPARK-27518][SQL][Web UI] Show statistics in Optimized Logical Plan in the "Details" On SparkSQL ui page when CBO is enabled Apr 26, 2019
@pengbo pengbo closed this Sep 5, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants