Task alerts - #721
Conversation
|
Alert if a task is killed due to slave decommissioning.
|
There was a problem hiding this comment.
instead of calculating this yourself, you should use averageRuntimeMillis in the SingularityDeployStatistics object, which can be grabbed from the deploy history endpoint (.../api/history/request/REQUEST_ID/deploy/DEPLOY_ID). it would also be wise to surface warnIfScheduledJobIsRunningPastNextRunPct from https://github.com/HubSpot/Singularity/blob/master/SingularityService/src/main/java/com/hubspot/singularity/config/SingularityConfiguration.java#L172 into the window.config object so that we don't have to hardcode the threshold value
|
Updated |
There was a problem hiding this comment.
i don't think this is necessary -- using the value inside SingularityConfiguration is probably good enough
There was a problem hiding this comment.
technically we could only make this call only if it's needed (i.e. if the task is scheduled + currently running)
2 participants
Show an alert on the task page if it is a scheduled task that has been running for longer than twice the average for tasks in the request. /cc @tpetr