You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Added
Add FluxSourceCriticalFailed, paging 24/7 on GitRepository in flux-giantswarm — the source an oncaller needs to push a change through Flux at any hour. GitRepository is removed from FluxSourceFailed in exchange, and that alert keeps working hours: an OCIRepository or HelmRepository that cannot be pulled says little about whether the service is running, and can wait for morning.
Add FluxMetricsMissing alert firing when source-controller stops emitting reconcile metrics for 30m, i.e. Flux is not running or not reconciling. Reads gotk_reconcile_duration_seconds_count, published by the controller itself, rather than gotk_resource_info, which comes from the flux-ksm kube-state-metrics instance and so reports KSM's health rather than Flux's.
Add team Bumblebee's agent platform availability alerts: AgentPlatformHelmReleaseNotReady (a team=bumblebee HelmRelease in flux-giantswarm not Ready for 1h), KagentControllerDown (kagent-controller without an available replica for 30m) and AgentPlatformPostgresClusterNotHealthy (a CloudNativePG Cluster in agent-platform or kagent outside its healthy phase for 30m). All three carry all_pipelines: "true": the agent platform's own management clusters include testing-pipeline installations, where the existing FluxGiantswarmHelmReleaseFailed and DeploymentNotSatisfiedBumblebee alerts fire but only reach Slack. The first two step back wherever those generic alerts page (stable and stable-testing pipelines), so a stuck component never opens two incidents.
Changed
Cut for: 2h to 30m on FluxSourceFailed and FluxWorkloadClusterSourceFailed, and add the cancel_if_kube_state_metrics_down, cancel_if_metrics_broken and cancel_if_monitoring_agent_down inhibitions so a monitoring blackout does not double-page.