-
Notifications
You must be signed in to change notification settings - Fork 587
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
idle job killer thinks job flows running Hive are idle #60
Comments
So, the step that runs hive reports that it's "done":
Probably the easiest fix is to leave alone job flows where the last step (and in most cases, the only step) isn't running the Hadoop streaming jar. Not sure what happens if you try to run streaming jobs in a job flow where you're already running Hive. Going to try it! |
It looks like the streaming job works, until you run a Hive query, and that kills the streaming job. So they don't mix in any practical way. So it looks like we should leave alone a job that has any non-streaming jar steps (since |
Huh, so here's what a Hadoop Debugging step looks like (which is compatible with hadoop streaming):
I think I'll only leave alone job flows that have no hadoop streaming steps, since they're clearly not the purview of |
Fixed in dev, and heavily tested! |
Fixed in v0.2.0. |
The idle job killer wrongly kills job flows that are running Hive-interactive. Need to fix this!
The text was updated successfully, but these errors were encountered: