[Feature] Dispatcher-saturation + mailbox-depth histograms

[pathosDev]

Size / Priority

• Size: M

Rationale

Today's metrics for dispatch include counters (actor_messages_delivered_total) and per-handler latency histogram (actor_message_handler_seconds). What's missing:

• Mailbox depth distribution: most actors have mailbox depth 0-1; some have spikes to 1000+. A histogram reveals the tail.
• Dispatcher saturation: ratio of "time the dispatcher was processing" vs "time available". 100% = no idle headroom; OOM/latency risk imminent.

Both are critical production-debugging signals. Without them, "the system is slow" diagnosis goes through pinning mailbox-by-mailbox.

Design sketch

// src/metrics/Metrics.ts — new metric families

// Per-actor mailbox depth (sampled per dispatch)
'actor_mailbox_depth' (histogram, label: 'actor.path')

// Dispatcher saturation (per-node, per-dispatcher)
'dispatcher_saturation_ratio' (gauge, label: 'dispatcher')

// Time spent waiting in mailbox before handler started
'actor_mailbox_wait_seconds' (histogram)

Hooks in ActorCell._dispatchOne:

// At dispatch start:
const queueEntryTs = env.enqueuedAt;
metrics.histogram('actor_mailbox_wait_seconds', {}, {...}).observe((performance.now() - queueEntryTs) / 1000);
metrics.histogram('actor_mailbox_depth', { 'actor.path': bucketize(this.path) }, {...}).observe(this.mailbox.size);

// Dispatcher saturation: track in the per-dispatcher event loop wrapper:
// busyTime / (busyTime + idleTime) sampled every N seconds.

Integration

• Envelope: extend with enqueuedAt: number (filled at tell time).
• ActorCell: existing instrumentation extended.
• Metrics.registry: respect [Security] Prometheus cardinality attack via user-controlled label values #131's cardinality cap (actor.path label → bucketize).
• No-op default: existing default metrics registry (NoOpRegistry) preserved for users who don't want metrics.

Out of scope / non-goals

• Tracing-style per-message attribution: out of scope; use the tracing extension for that.
• Backpressure auto-tuning based on saturation: nice future feature ([Feature] ML auto-tuning of cluster parameters #213).

Open design questions

1. Sampling frequency: every dispatch (sketch) vs every Nth. Recommend every dispatch — cheap with the existing histogram.
2. Saturation measurement: requires event-loop instrumentation. Node has perf_hooks.monitorEventLoopDelay; Bun has similar; Deno different. Cross-runtime adapter.
3. Cardinality: actor.path can explode (every entity has its own path). Bucketize to actor-class-name + parent-path; document.

Test plan

1. Stress test → mailbox-depth histogram p99 reflects build-up.
2. Idle system → saturation ~0%.
3. Saturated system → saturation ~100%.
4. Cardinality cap ([Security] Prometheus cardinality attack via user-controlled label values #131) honoured.
5. Cross-runtime: Bun, Node, Deno.

Acceptance criteria

• actor_mailbox_depth + actor_mailbox_wait_seconds histograms.
• dispatcher_saturation_ratio gauge.
• Cross-runtime event-loop saturation measurement.
• Cardinality bucketize for actor.path.
• Documentation: "Diagnosing slow actors with mailbox metrics".
• Test suite (5 cases).
• CHANGELOG entry.