Skip to content

Commit

Permalink
[SPARK-22637][SQL] Only refresh a logical plan once.
Browse files Browse the repository at this point in the history
## What changes were proposed in this pull request?
`CatalogImpl.refreshTable` uses `foreach(..)` to refresh all tables in a view. This traverses all nodes in the subtree and calls `LogicalPlan.refresh()` on these nodes. However `LogicalPlan.refresh()` is also refreshing its children, as a result refreshing a large view can be quite expensive.

This PR just calls `LogicalPlan.refresh()` on the top node.

## How was this patch tested?
Existing tests.

Author: Herman van Hovell <hvanhovell@databricks.com>

Closes #19837 from hvanhovell/SPARK-22637.
  • Loading branch information
hvanhovell authored and gatorsmile committed Nov 29, 2017
1 parent a10b328 commit 475a29f
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -480,7 +480,7 @@ class CatalogImpl(sparkSession: SparkSession) extends Catalog {
if (tableMetadata.tableType == CatalogTableType.VIEW) {
// Temp or persistent views: refresh (or invalidate) any metadata/data cached
// in the plan recursively.
table.queryExecution.analyzed.foreach(_.refresh())
table.queryExecution.analyzed.refresh()
} else {
// Non-temp tables: refresh the metadata cache.
sessionCatalog.refreshTable(tableIdent)
Expand Down

0 comments on commit 475a29f

Please sign in to comment.