Health status of Jobs is incomplete

[ANeumann82]

What happened:
I'm currently working on the Cassandra Backup. This includes a plan that is triggered to execute the backups.
This plan deploys one or more jobs that do the backup. The are not supposed to run multiple times, therefore have a backoffLimit of 0. So when the backup process fails, the job has a status of Failed and will never complete.

KUDO currently waits for the Job to be healthy and have a status of Success. The plan never completes or fails and stays in IN_PROGRESS

What you expected to happen:
KUDO should acknowledge the fact that the job has failed and will not recover and set the step status to FATAL_ERROR with a descriptive error message.

How to reproduce it (as minimally and precisely as possible):

• Create a plan that deploys a Job that has backoffLimit of 0 and fails.

[ANeumann82]

Hmmm. I'm not sure if we should use FATAL_ERROR in this case. I think there's difference in an KUDO FATAL_ERROR and an unrecoverable error outside of KUDO. Maybe a new error code?