v0.30.0
Valkey Cluster V2 (Raft) health monitoring — quorum-loss and leadership-churn detection, plus a live Raft Health panel.
✨ Features
- Cluster V2 (Raft) health detection. The anomaly engine now understands Valkey's Raft-based clustering (
cluster-protocol raft) and watches the two failure modes that mode is uniquely prone to:- Quorum loss / no reachable leader (CRITICAL). Calibrated against a live 3-node Raft cluster: a node that has lost its majority keeps reporting
cluster_state:ok(its slots stay "covered") and never inflates its term (pre-vote), socluster_stateis not a reliable signal. The detector instead keys on the node repeatedly seeking a leader it can't elect (candidate/pre-candidate) while its commit index stays frozen. Detection windows are derived from the cluster'scluster-node-timeout, so they hold across any tuning, and a healthy failover or a one-off election blip is correctly ignored. Incidents auto-resolve when quorum returns and stay pinned (surviving a dismiss) while the outage is live. - Election churn (WARNING). Repeated completed elections (term climbing) in a short window flag flapping leadership from an unstable/overloaded primary or a split network.
- Quorum loss / no reachable leader (CRITICAL). Calibrated against a live 3-node Raft cluster: a node that has lost its majority keeps reporting
- Cluster Overview → "Cluster V2 · Raft Health" panel. Shows the connected node's Raft role, current term, commit / last-applied / log-entries progress (with an apply-lag callout), and the elected leader. It self-hides on legacy gossip clusters, surfaces "Electing" while a node is seeking a leader, distinguishes a slot/serving
cluster_state:failfrom a true "no quorum", and pins the live outage from the authoritative detector. - Critical "Raft cluster has lost quorum" banner on the Anomaly dashboard, with a Raft-specific remediation runbook (majority math, don't force-promote, don't wipe survivors), plus a friendly "Raft Health" metric label.
- Gossip-mode gating. On a Raft cluster the gossip-era topology detectors (duplicate-primary #2261, stuck-replica #2090) are skipped — topology is consensus-managed there — while remaining fully active on classic gossip clusters, even across transient
CLUSTER INFOblips. - OTLP telemetry mirroring (#319). The monitor can mirror its existing Prometheus metrics and discrete monitoring events to an OTLP endpoint. Opt-in and no-op unless
OTEL_EXPORTER_OTLP_ENDPOINT/OTEL_TELEMETRY_ENABLEDare set. - Agent Memory
consolidate()modes (#324).consolidate()now takes a requiredmode: 'summary' | 'facts', surfaces unmatched tombstones, and ships recall tuning. (Also released standalone asagent-memory-v0.7.0.)
⬆ Upgrade notes
- Fully backwards compatible; no config changes. On non-Raft (gossip/standalone) deployments nothing changes.
- Raft health detection runs inside the anomaly engine, which is a Pro/Enterprise feature — it activates with a valid license; Community builds are unaffected.
- OTLP mirroring is off by default — enable it only via the
OTEL_*env vars above. - Docker:
betterdb/monitor:0.30.0
Full Changelog: v0.29.0...v0.30.0
What's Changed
- feat(otel): mirror Prometheus metrics + discrete events to OTLP by @jamby77 in #319
- feat(agent-memory): merge consolidate() modes, surface unmatched tombstones, tune recall by @KIvanow in #324
- feat(anomaly): Raft cluster (Cluster V2) health detection + gossip-mode gating by @KIvanow in #326
Full Changelog: v0.29.0...v0.30.0