You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am evaluating sparkMeasure for my use case but I always gets jobID filed as "0" or a series of 1. Is this expected behavior or I am missing something here?
Below are the outputs of a spark2-submit pi job with sparkMeasure and from a spark-shell job resp.
Also, how can we figure out to which task a particular metric belongs to?
Thanks
Amit
The text was updated successfully, but these errors were encountered:
Amittyagi007
changed the title
jobId filed comes as "0" in stage and task Metrics.
jobId field only comes as "0" or "1" in stage and task Metrics.
Jan 7, 2019
Hi,
I am evaluating sparkMeasure for my use case but I always gets jobID filed as "0" or a series of 1. Is this expected behavior or I am missing something here?
Below are the outputs of a spark2-submit pi job with sparkMeasure and from a spark-shell job resp.
+-----+--------+-------+--------------------+
|jobId|jobGroup|stageId| name|
+-----+--------+-------+--------------------+
| 0| null| 0|reduce at SparkPi...|
+-----+--------+-------+--------------------+
+-----+--------+
|jobId|jobGroup|
+-----+--------+
| 0| null|
| 1| null|
| 1| null|
| 1| null|
| 1| null|
| 1| null|
| 1| null|
Also, how can we figure out to which task a particular metric belongs to?
Thanks
Amit
The text was updated successfully, but these errors were encountered: