You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
scala> val taskMetrics = ch.cern.sparkmeasure.TaskMetrics(spark)
taskMetrics: ch.cern.sparkmeasure.TaskMetrics = TaskMetrics(org.apache.spark.sql.SparkSession@69c0bae6,false)
scala> taskMetrics.runAndMeasure
def runAndMeasure[T](f: => T): T
scala> taskMetrics.runAndMeasure(spark.sql("select * from test.test"))
Hive Session ID = 04e7280b-0a45-4fad-867f-f1447faf6bf4
Time taken: 4895 ms
19/02/20 09:01:58 WARN Utils: Truncated the string representation of a plan since it was too large. This behavior can be adjusted by setting 'spark.debug.maxToStringFields' in SparkEnv.conf.
19/02/20 09:01:58 WARN TaskMetrics: Stage metrics data refreshed into temp view PerfTaskMetrics
scala.MatchError: (elapsedTime,null) (of class scala.Tuple2)
at ch.cern.sparkmeasure.TaskMetrics$$anonfun$report$1.apply(taskmetrics.scala:206)
at ch.cern.sparkmeasure.TaskMetrics$$anonfun$report$1.apply(taskmetrics.scala:206)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:186)
at ch.cern.sparkmeasure.TaskMetrics.report(taskmetrics.scala:206)
at ch.cern.sparkmeasure.TaskMetrics.printReport(taskmetrics.scala:215)
at ch.cern.sparkmeasure.TaskMetrics.runAndMeasure(taskmetrics.scala:282)
... 49 elided
What I'm doing wrong?
The text was updated successfully, but these errors were encountered:
Hi. I'm trying use this library.
Samples of code:
build.sbt
spark-shell
What I'm doing wrong?
The text was updated successfully, but these errors were encountered: