monitoring

Pre-Alpha. This page describes behavior that may change.

Ze gives you three different ways to watch a running daemon: an auto-refreshing peer dashboard for the operator at the keyboard, a live event stream for the engineer chasing a bug, and a Prometheus endpoint for everything else. They all read the same in-process state, so a number you see in the dashboard matches a metric you scrape from Prometheus.

The live peer dashboard

ze cli monitor bgp

Auto-refreshing every two seconds. Sortable, color-coded peer table with router identity, state, uptime, and update rates. Navigate with j and k, sort with s or S, hit Enter to drop into a peer's detail page, and Esc to exit.

This is the screen you leave open on a side monitor when you are about to touch something risky. It is also the fastest way to confirm at a glance that a session you just brought up is actually moving messages.

The event stream

ze cli monitor event

The raw firehose. Filter it before it overwhelms you.

ze cli monitor event peer upstream event update direction received
ze cli monitor event event state                # Just up and down

The filters compose. peer <selector> narrows to one peer, event <type>[,<type>] selects which event types to include, and direction received or direction sent narrows to one side of the conversation. There is no exclude capability; you specify which types to include via event. The recognised event types are update, open, notification, keepalive, refresh, state, negotiated, eor, and rpki.

For scripts, pipe the stream through | json and parse it. Every event uses the same envelope: a peer block, a message block with id, direction, and type, and the per-event payload (update carries next-hop, as-path, local-preference, and the per-family NLRI lists; state carries the new FSM state).

Prometheus

Ze exposes a Prometheus endpoint when you set telemetry { prometheus { ... } } in the config. Metrics refresh every ten seconds.

The metrics you actually need most of the time are these.

Metric	Why you care
ze_peer_state{peer}	3 means Established. Anything else is a graphable problem.
ze_peer_messages_received_total{peer,type}	Update rate, broken down by message type.
ze_peer_messages_sent_total{peer,type}	Same for outbound.
ze_bgp_prefix_count{peer,family}	Current per-family prefix count for a peer.
ze_bgp_prefix_ratio{peer,family}	count / maximum. Alert above 0.9.
ze_bgp_prefix_warning_exceeded{peer,family}	1 if the warning threshold has been crossed.
ze_bgp_pool_used_ratio	Forwarding pool utilisation. Anything above 0.8 means you are close to congestion teardown.
ze_uptime_seconds	The reactor's uptime.
ze_info{version,router_id,local_as}	Tag for joins.

There are also histograms for connection timing (ze_peer_dial_seconds, ze_peer_connect_attempt_seconds, ze_peer_backoff_seconds), per-peer overflow counters, and prefix-limit teardown counters. The full list lives in the in-tree monitoring guide.

Plugin event subscription

Plugins are first-class consumers of the same event stream. A plugin binding under a peer declares which events it wants:

peer upstream {
    process my-plugin {
        receive [ update state ]
    }
}

The plugin gets each event through its OnEvent callback. The set of recognised event types is the same as the CLI filter list, plus any extra types a plugin registers itself.

OS metrics (Netdata-compatible)

Ze includes Netdata-compatible OS collectors that expose system metrics through Prometheus. These are enabled automatically when the telemetry prometheus block is configured.

Collector	Metrics
CPU	Per-core user/system/idle/iowait/steal, load averages, context switches.
Memory	Total, available, buffers, cached, swap, slab, hugepages.
Network	Per-interface rx/tx bytes/packets/errors/drops, carrier state.
Disk	Per-device reads/writes/io-time, filesystem usage.
Conntrack	Entries, max, searches, found, new, invalid, expect.
IPv6	Per-interface in/out/forwarded/discarded counts.
Softirqs	Per-CPU softirq counts by type.
Sockstat6	IPv6 socket statistics.

Host hardware metrics

When the host inventory plugin is active, hardware metrics are exposed under the ze_host_* namespace:

Metric	Description
ze_host_memory_*	DIMM size, ECC errors, type.
ze_host_cpu_*	Core count, frequency, model.
ze_host_nic_*	Speed, carrier, firmware version.
ze_host_thermal_*	Temperature readings per sensor.
ze_host_storage_*	SMART health status, capacity, wear level.

Continuous monitoring commands

monitor ping

monitor ping <target>
monitor ping <target> interval 500ms

Continuous ICMP ping with live statistics: Sent, Recv, Loss%, Last, Min, Avg, Max, StDev. Use | log for scrollback output suitable for correlation with other events.

monitor traceroute

monitor traceroute <target>
monitor traceroute <target> | log | origin
monitor traceroute <target> | log | resolve

mtr-style continuous traceroute with per-hop loss and latency statistics. | log | origin appends one line per round and annotates hops with ASN names. | log | resolve adds reverse DNS hostnames.

monitor system netlink

monitor system netlink all
monitor system netlink link
monitor system netlink route

Streams live kernel events as JSON. Replaces ip monitor. Useful for observing route additions/deletions, link state changes, and address changes in real time.

monitor interface rate

monitor interface rate

Live per-interface rate tracking (rx/tx bytes, packets, errors, drops).

Live packet capture

show capture interface eth0 tcp port 179 count 10 format text
show capture interface eth0 tcp port 179 duration 10s format pcap

Replaces tcpdump. Uses AF_PACKET with BPF filters. Output in text or pcap format.

Diagnostics

Ze includes runtime diagnostics for production debugging. See Production Diagnostics for the full symptom-based guide.

Feature	CLI
BGP FSM history	Global event ring and per-peer state transition history.
L2TP FSM history	Per-session event ring with disconnect reasons.
Capture rings	BGP, L2TP, and BFD packet capture rings, pcap export.
VPP trace	VPP per-node packet trace capture.
TCP connectivity	show tcp-check <ip> <port> for port probing.
Path tracing	show traceroute <target> for single-shot ICMP trace.
Socket state	show system sockets replaces ss.
Kernel log	show system kernel-log replaces dmesg.
Goroutine dump	show system goroutines for leak detection.
FD monitoring	show system file-descriptors for exhaustion detection.
Profiling	show system profile cpu/heap for pprof profiles.
DNS diagnostics	show dns lookup/cache for DNS resolution and cache inspection.
Crash capture	show crashes for enriched panic reports (build/runtime metadata, ring-buffer context).
SMART disk health	show storage smart for per-device health, temperature, and self-test schedule.
PKI certificates	show pki certificate <name> pem / bundle pem / fingerprint for export and inspection.
Structured log query	Slog ring buffer query for recent structured log entries.
Component health	show health and /health endpoint with per-component status (bgp, fib, firewall, iface, plugins, vpp, ipsec, pki, l2tp).

See also

• Show commands for the read-only snapshot view.
• Production Diagnostics for symptom-based troubleshooting.
• Health Checks for the health registry and ze doctor.
• Plugins for the event-bus consumer model.
• Logging for log levels and backends.

Adapted from main/docs/guide/monitoring.md.