Skip to content

dashboard

Thomas Mangin edited this page Jul 25, 2026 · 2 revisions

Pre-Alpha. This page describes behavior that may change.

The chaos dashboard is a live web UI for ze-chaos. It shows peers, routes, convergence, chaos events, and property verification in real time, with an interactive control panel for pausing the chaos scheduler, adjusting the rate, and triggering manual faults. It is built with HTMX and Server-Sent Events, with every asset embedded in the binary via go:embed.

Chaos dashboard overview

Starting it

The dashboard is built into ze-chaos and exposed on a local HTTP port.

ze-chaos --web :9000 --seed 42 --peers 40 --duration 5m | ./bin/ze -

Open http://localhost:9000/ in a browser. No authentication, no TLS. The dashboard is for local operator use during a run, not for remote exposure.

The layout

Three panels across a desktop monitor.

Header bar. Run status badge (RUNNING / COMPLETED / FAILED), the current seed (copyable, so you can paste it into --replay later), an elapsed clock, the peer gauge ("N of M established"), and the route gauge (announced, received, and deltas). Every second via an SSE tick event.

Left sidebar. Summary cards, property badges, and the control panel.

  • Summary cards. Peers (established and total, iBGP and eBGP counts), Routes (announced, received, missing in red, extra in red), Chaos (events fired, reconnections, withdrawn routes), and Convergence (min, average, max, p99 latency).
  • Property badges. One per active property. Green means PASS, red means FAIL, grey means not checked. Click a red badge to expand the violation detail inline.
  • Control panel. A chaos toggle, a chaos-rate slider (0.0 to 1.0), a manual fault trigger (action type and target peer), a "new seed" input that restarts the run, and a stop button.

Main content. The peer table, a peer detail pane that opens when you click a row, and a visualisation tab strip.

The peer table

With 200 or more mock peers, showing them all at once becomes useless. The table shows an active set of at most 40 peers (configurable via --max-visible) and uses three categories to decide what is visible.

Category Behaviour Removal
Pinned Always visible, toggled via a pin icon. Manual unpin only.
Auto-promoted Appears when a noteworthy event fires (state change, disconnect, large route delta). Decays after an adaptive TTL expires.
Manually added You pick a peer from a search dropdown. Manual remove, or decay if not pinned.

Auto-promotion uses event priority to decide which peers deserve a slot. Disconnects, FSM errors, and large route deltas score highest. Idle peers decay out naturally.

The visualisation tabs

Each tab is an HTMX fragment loaded on demand. Five tabs.

  • Events. A live scrolling list of every event the reporter has seen. Filterable by peer, event type, and direction.
  • Peer timeline. Per-peer horizontal timeline showing state transitions, chaos triggers, and route deltas. Useful for "when did this peer actually break?"
  • Convergence. A histogram of convergence latencies, plus min, average, max, and p99.

Chaos dashboard convergence

  • Chaos timeline. When each chaos event fired, on which peer, and in which category.
  • Route matrix. A visualisation of which routes are known to which peers. Missing cells are red and are the first thing you should look at when a convergence property fails.

Property verification at a glance

The property badges in the sidebar are the first thing to look at. A run where all badges stay green the whole way through is a run that passed. A run where a badge flips red partway through is a run that found a bug, and the badge is the starting point for the investigation.

Click the red badge to expand the violation detail. The detail card names the property, the peer involved, the event that triggered the failure, and the timestamp. From there, pause the run, open the peer detail pane for the affected peer, and look at the event timeline.

The controls

The control panel is the difference between "watch a run" and "drive a run". Three interactions matter.

Pause the chaos scheduler. The toggle at the top of the control panel stops new faults from firing without stopping the run. The session continues, convergence can finish, and you can look at the state without new faults complicating the picture.

Adjust the rate live. The slider goes from 0.0 (no faults) to 1.0 (fault every operation). Moving it does not restart the run. This is the fastest way to turn the heat up or down while watching the dashboard.

Trigger a manual fault. Pick an action from the dropdown and a target peer from the table, and hit the button. The orchestrator queues the fault for the next event loop tick. This is how you reproduce a specific fault on a specific peer without depending on the PRNG.

How it plugs in

The dashboard is implemented as a report.Consumer, the same interface used by the terminal dashboard, the JSON log, and the Prometheus exporter. It plugs into the same event fan-out. No changes to the Reporter multiplexer were needed to add it. That means every consumer sees the same events at the same time, and the dashboard is never a different view of reality.

The one constraint is that ProcessEvent() runs synchronously on the main event loop. It must update internal state and push onto a broadcast channel, and it must never block on HTTP or template rendering. That is why the templates render in the SSE broker goroutine, not in the consumer callback.

See also

Adapted from main/docs/architecture/chaos-web-dashboard.md.

Home

About

First Steps

Configuration

Operation

Interfaces

Plugins

Plugin Development

Chaos Testing

Blueprints

Development

Reference

Clone this wiki locally