Skip to content

Commit 33eda21

Browse files
pan3793ulysses-you
authored andcommitted
[KYUUBI #1884] Fix compile SQL measurement
### _Why are the changes needed?_ The previous measurement of parsing and analysis does not make sense because `Dataset` calls `queryExecution.assertAnalyzed()` in the constructor. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #1884 from pan3793/measure. Closes #1884 8da9387 [Cheng Pan] Fix compile SQL measurement Authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: ulysses-you <ulyssesyou@apache.org>
1 parent be6133a commit 33eda21

File tree

1 file changed

+7
-13
lines changed
  • dev/kyuubi-tpcds/src/main/scala/org/apache/kyuubi/tpcds/benchmark

1 file changed

+7
-13
lines changed

dev/kyuubi-tpcds/src/main/scala/org/apache/kyuubi/tpcds/benchmark/Query.scala

Lines changed: 7 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ import scala.collection.mutable.ArrayBuffer
2222
import scala.language.implicitConversions
2323

2424
import org.apache.spark.sql.DataFrame
25+
import org.apache.spark.sql.catalyst.QueryPlanningTracker
2526
import org.apache.spark.sql.catalyst.analysis.UnresolvedRelation
2627
import org.apache.spark.sql.execution.SparkPlan
2728

@@ -66,19 +67,12 @@ class Query(
6667
try {
6768
val dataFrame = buildDataFrame
6869
val queryExecution = dataFrame.queryExecution
69-
// We are not counting the time of ScalaReflection.convertRowToScala.
70-
val parsingTime = measureTimeMs {
71-
queryExecution.logical
72-
}
73-
val analysisTime = measureTimeMs {
74-
queryExecution.analyzed
75-
}
76-
val optimizationTime = measureTimeMs {
77-
queryExecution.optimizedPlan
78-
}
79-
val planningTime = measureTimeMs {
80-
queryExecution.executedPlan
81-
}
70+
queryExecution.executedPlan
71+
val phases = queryExecution.tracker.phases
72+
val parsingTime = phases(QueryPlanningTracker.PARSING).durationMs.toDouble
73+
val analysisTime = phases(QueryPlanningTracker.ANALYSIS).durationMs.toDouble
74+
val optimizationTime = phases(QueryPlanningTracker.OPTIMIZATION).durationMs.toDouble
75+
val planningTime = phases(QueryPlanningTracker.PLANNING).durationMs.toDouble
8276

8377
val breakdownResults =
8478
if (includeBreakdown) {

0 commit comments

Comments
 (0)