Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jobId field only comes as "0" or "1" in stage and task Metrics. #22

Closed
Amittyagi007 opened this issue Jan 7, 2019 · 0 comments
Closed

Comments

@Amittyagi007
Copy link

Hi,

I am evaluating sparkMeasure for my use case but I always gets jobID filed as "0" or a series of 1. Is this expected behavior or I am missing something here?

Below are the outputs of a spark2-submit pi job with sparkMeasure and from a spark-shell job resp.

+-----+--------+-------+--------------------+
|jobId|jobGroup|stageId| name|
+-----+--------+-------+--------------------+
| 0| null| 0|reduce at SparkPi...|
+-----+--------+-------+--------------------+

+-----+--------+
|jobId|jobGroup|
+-----+--------+
| 0| null|
| 1| null|
| 1| null|
| 1| null|
| 1| null|
| 1| null|
| 1| null|

Also, how can we figure out to which task a particular metric belongs to?

Thanks
Amit

@Amittyagi007 Amittyagi007 changed the title jobId filed comes as "0" in stage and task Metrics. jobId field only comes as "0" or "1" in stage and task Metrics. Jan 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant