Ideas: What should the Phase 5 Grafana dashboard look like?

[web3guru888]

The question

Phase 5 adds 20+ new Prometheus metrics (Issue #220). What should the Grafana dashboard for Phase 5 look like? This discussion collects panel ideas and layout options before someone writes docs/grafana/phase5.json.

---

Context: What Phase 5 operators need to see

The Phase 5 monitoring surface has four distinct concerns:

1. Online learning health — is STDP updating weights safely? Is hot-reload succeeding?
2. Emergent coordination — are AgentMesh coalitions forming and dissolving normally?
3. Memory consolidation — is SLEEP_PHASE running? Are KG writes balanced across tiers?
4. Rollback state — is any rollback active, and for how long?

Each concern maps to a different alerting posture: learning health is safety-critical, coordination is performance, memory is latency-sensitive, rollback is incident-response.

---

Option A: Four-row layout (one row per pillar)

Row 1: ONLINE LEARNING
  [Weight delta norm P95 — time series]  [Hot-reload success rate]  [Rejection rate gauge]

Row 2: EMERGENT COORDINATION
  [Active coalitions — stat]  [Coalition size histogram]  [Pheromone strength — sparkline]

Row 3: PERSISTENT MEMORY
  [KG writes by tier — stacked bar]  [Sleep-phase active — state timeline]  [Conflict rate]

Row 4: ROLLBACK MANAGER
  [Rollback active by scenario — table]  [Ticks remaining — gauge]  [Recovery count — stat]

Pro: Clean separation of concerns. Each row tells a single story.
Con: 12+ panels is a lot of vertical scrolling. Needs folding rows.

---

Option B: Safety-first layout (critical metrics prominent)

TOP: Safety invariant status
  [Weight delta norm vs. WARNING/CRITICAL thresholds — reference lines]
  [Rollback active — alert panel]  [Safety gate rejection rate — gauge + color]

MIDDLE: Learning & coordination summary
  [Hot-reload success rate]  [Active coalitions]  [Phi score trend]

BOTTOM: Memory & ops detail
  [KG write tiers]  [Sleep phase timeline]  [Planner latency P95]

Pro: Incident responders see rollback status and safety violations immediately.
Con: Mixes concerns in the middle section, harder to tell a coherent story.

---

Option C: Single-pane "Phase 5 health score" + drilldown rows

TOP: Composite health score (0-100)
  Weighted average of: rejection_rate_ok (30%) + hot_reload_ok (20%) + coalitions_ok (20%) + rollback_free (30%)

DRILLDOWN (collapsed by default):
  Row A: Online Learning detail
  Row B: Coordination detail
  Row C: Memory detail
  Row D: Rollback history

Pro: Single number for NOC dashboards and SLO reporting. Drilldowns for on-call engineers.
Con: The composite formula is opinionated and may hide real problems.

---

Recommendation

Option A for the initial template, with an alert panel at the very top showing phase5_rollback_active for any scenario. Rationale:

• Matches the four-pillar structure already used in the wiki and Phase 5 issue tracker
• Easier for contributors to maintain (each section is self-contained)
• Alert panel addresses the Option B concern about rollback visibility

The composite score (Option C) can be a separate "SLO" dashboard once the system has enough production data to calibrate weights.

---

Call to action

If you have Grafana experience or opinions on dashboard layout — share them here before the docs/grafana/phase5.json file gets written. What panels would you add or remove?

---

Related

• Issue Add Phase 5 Prometheus metrics layer: learning quality, safety invariant status, and rollback state #220: Phase 5 Prometheus metrics spec
• Discussion Show & Tell: Inside Phase5MetricsExporter — Prometheus bridge design, 20-metric taxonomy, and bucket strategy #221: Phase5MetricsExporter design
• Discussion Ideas: What alerting strategy should Phase 5 use? Fixed thresholds vs. sustained windows vs. adaptive Bollinger bands #218: Alerting strategy (sustained window vs. Bollinger bands)
• Issue Add Phase 4.3 production deployment: Docker Compose, Kubernetes Helm chart, Prometheus metrics, and health endpoints #164: Phase 4.3 base Grafana setup