Skip to content

v4.119.0

Choose a tag to compare

@taylorbot taylorbot released this 06 Sep 09:23
· 29 commits to main since this release
0ca4d77

Added

  • Add FluxSourceCriticalFailed, paging 24/7 on GitRepository in flux-giantswarm — the source an oncaller needs to push a change through Flux at any hour. GitRepository is removed from FluxSourceFailed in exchange, and that alert keeps working hours: an OCIRepository or HelmRepository that cannot be pulled says little about whether the service is running, and can wait for morning.
  • Add FluxMetricsMissing alert firing when source-controller stops emitting reconcile metrics for 30m, i.e. Flux is not running or not reconciling. Reads gotk_reconcile_duration_seconds_count, published by the controller itself, rather than gotk_resource_info, which comes from the flux-ksm kube-state-metrics instance and so reports KSM's health rather than Flux's.
  • Add team Bumblebee's agent platform availability alerts: AgentPlatformHelmReleaseNotReady (a team=bumblebee HelmRelease in flux-giantswarm not Ready for 1h), KagentControllerDown (kagent-controller without an available replica for 30m) and AgentPlatformPostgresClusterNotHealthy (a CloudNativePG Cluster in agent-platform or kagent outside its healthy phase for 30m). All three carry all_pipelines: "true": the agent platform's own management clusters include testing-pipeline installations, where the existing FluxGiantswarmHelmReleaseFailed and DeploymentNotSatisfiedBumblebee alerts fire but only reach Slack. The first two step back wherever those generic alerts page (stable and stable-testing pipelines), so a stuck component never opens two incidents.

Changed

  • Cut for: 2h to 30m on FluxSourceFailed and FluxWorkloadClusterSourceFailed, and add the cancel_if_kube_state_metrics_down, cancel_if_metrics_broken and cancel_if_monitoring_agent_down inhibitions so a monitoring blackout does not double-page.