[GLUTEN-3559][VL] Fixed unit test failing after change of Spark-38674 #4143
Conversation
|
Run Gluten Clickhouse CI |
1 similar comment
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
@JkSelf Kindly review. |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
@zhli1142015 @ulysses-you Could you please help in reviewing this? |
gluten-core/src/main/scala/org/apache/spark/sql/execution/ColumnarSubqueryBroadcastExec.scala
Outdated
Show resolved
Hide resolved
| @@ -550,17 +606,26 @@ class GlutenDynamicPartitionPruningV1SuiteAEOff | |||
| scanOption.get | |||
| } | |||
|
|
|||
| def getDriverMetrics(plan: SparkPlan, key: String): Option[SQLMetric] = { | |||
There was a problem hiding this comment.
Looks this function is duplicated several times. If any reason we need it as we always pass scan for plan.
There was a problem hiding this comment.
@zhli1142015 the corresponding metrics need to be sourced from driverMetrics (as against metrics, used earlier)
gluten-ut/spark34/src/test/scala/org/apache/spark/sql/GlutenDynamicPartitionPruningSuite.scala
Show resolved
Hide resolved
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
|
Run Gluten Clickhouse CI |
What changes were proposed in this pull request?
This PR fixes the following suites:
GlutenDynamicPartitionPruningV1SuiteAEOff
GlutenDynamicPartitionPruningV1SuiteAEOff
GlutenDynamicPartitionPruningV1SuiteAEOn
GlutenDynamicPartitionPruningV1SuiteAEOnDisableScan
GlutenDynamicPartitionPruningV1SuiteAEOffDisableScan
GlutenDynamicPartitionPruningV2SuiteAEOff
GlutenDynamicPartitionPruningV2SuiteAEOn
GlutenDynamicPartitionPruningV2SuiteAEOnDisableScan
GlutenDynamicPartitionPruningV2SuiteAEOffDisableScan
(Fixes: #3559)
How was this patch tested?
Ran these UTs to confirm the fix