Reduce alert noise during cascading failures

[scotwells]

Summary

Adds alert inhibition rules so that when a root-cause alert fires, its predictable downstream symptoms are suppressed. This prevents alert storms during infrastructure failures.

What's included

An AlertmanagerConfig resource (prometheus-compatible, auto-converted by the VM operator) with 10 inhibition rules:

• Severity-based: Critical alerts suppress related warnings for the same component
• Apiserver down: Suppresses error rate and latency alerts
• ClickHouse unavailable: Suppresses query latency and pipeline stall alerts
• All NATS sources stalled: Suppresses individual consumer alerts
• Processor down: Suppresses generation stalled, error rate, and all DLQ alerts
• NATS disconnected: Suppresses generation stalled
• Keeper session errors: Suppresses ZooKeeper exception alerts
• Pipeline backlog critical: Suppresses consumer lag warning
• SLO page-burn: Suppresses equivalent threshold alerts to avoid duplicates

Validated in staging

• AlertmanagerConfig accepted and auto-converted by the VM operator
• All 10 inhibition rules loaded into alertmanager config (verified via config secret)
• Required a minimal route/receiver to pass VM operator CRD validation

Test plan

• Deploy to staging and verify resource is accepted
• Confirm inhibition rules appear in alertmanager generated config
• Simulate a critical alert and verify downstream warnings are suppressed

Closes #146

🤖 Generated with Claude Code