Alerts integration improvements and oom killer #9
Conversation
… + add oom killer enricher
playbooks/oom_killer.py
Outdated
| for pod in results.items: | ||
| oom_statuses = filter(is_oom_status, pod.status.containerStatuses) | ||
| for status in oom_statuses: | ||
| dt = datetime.fromisoformat(status.lastState.terminated.finishedAt.replace("Z", "+00:00")) |
There was a problem hiding this comment.
isn't there a way to use the k8s datetime str?
I think Z means utc.
If there will be an api server running on a different tz, I think there will be a bug here
this maybe:
dt = datetime.datetime.strptime(status.lastState.terminated.finishedAt, "%Y-%m-%dT%H:%M:%S%z")
There was a problem hiding this comment.
So this is a little tricky and it's similar to the Hikaru issue we saw with datetimes.
All the datetimes I get from the APIServer have Z at the end without a specific timezone set. I've done some brief research and I'm pretty sure that APIServers typically run in UTC and changing that is extremely uncommon / impossible.
Are you sure that the code you suggested behaves differently then what I committed? Wont both snippets throw an exception if the string that K8s returns has a timezone present?
I'm ok just ignoring this for now as I'm not sure the issue is actually possible. Wdyt?
| attribute_name: str | ||
| prometheus_label: str | ||
|
|
||
| MAPPINGS = [ResourceMapping(RobustaPod, "pod", "pod"), |
There was a problem hiding this comment.
Does the order matters?
I thought it did matter, now I'm not sure
I think some alerts may contain more than a single attribute (like pod and job)
There was a problem hiding this comment.
Yes, they can have multiple attributes, but the order doesn't matter as I will populate all of them.
|
added some comments, left for you to merge |
* updated forwarder and runner serviceaccount permission for clusterrolebindings * Job Action Added * Webex Sink Integration (#3) * Added webex_sink * Bug fix with fileblocks, code refactoring, documentation add and minor bug fixes * Delete job_restart_on_oomkilled.py * Updated Webex_Sink (#6) * Added webex_sink * Bug fix with fileblocks, code refactoring, documentation add and minor bug fixes * Added Enums and replaced emoji code with emoji icons * Removed PDF functionality and separated JSON logic from adaptive card function. (#7) * Added webex_sink * Bug fix with fileblocks, code refactoring, documentation add and minor bug fixes * Added Enums and replaced emoji code with emoji icons * Removed PDF functionality and seperated JSON creation logic from the adaptivecard function logic * Formatted using Black * Changes in _send_files and _separate_blocks function (#8) * Added webex_sink * Bug fix with fileblocks, code refactoring, documentation add and minor bug fixes * Added Enums and replaced emoji code with emoji icons * Removed PDF functionality and seperated JSON creation logic from the adaptivecard function logic * Formatted using Black * Changes in _send_files and _separate_blocks function * Changes in _send_files and _separate_blocks function (#9) * Added webex_sink * Bug fix with fileblocks, code refactoring, documentation add and minor bug fixes * Added Enums and replaced emoji code with emoji icons * Removed PDF functionality and seperated JSON creation logic from the adaptivecard function logic * Formatted using Black * Changes in _send_files and _separate_blocks function Co-authored-by: wahajXgrid <wahaj.ali@xgrid.co> Co-authored-by: wahajXgrid <107921443+wahajXgrid@users.noreply.github.com>
No description provided.