Where
docs/src/content/docs/operations/troubleshooting.mdx (all four PromQL blocks)
docs/src/content/docs/operations/tuning/dispatcher-tuning.mdx
docs/src/content/docs/operations/tuning/mailbox-sizing.mdx
docs/src/content/docs/operations/overview.mdx
docs/src/content/docs/operations/upgrades/rolling-migration.md
docs/src/content/docs/observability/management/http-endpoints.mdx (sample scrape output)
- the German mirrors of all of the above under
docs/src/content/docs/de/
What is wrong
The operations section is the runbook an on-call engineer opens at 3am. All four of its PromQL blocks query a metric family that src/ never creates. Against a real actor-ts process every one of them returns an empty result — which in Grafana, and in an alert rule, is indistinguishable from "the system is healthy".
The complete set of metric names emitted anywhere in src/ is eight. Every emission site, verbatim:
src/internal/ActorCell.ts:801-804
metricsOf(this.system).counter(
'actor_created_total', {},
{ help: 'Cumulative count of actors successfully started.' },
).inc();
src/internal/ActorCell.ts:889-892
metricsOf(this.system).counter(
'actor_terminated_total', {},
{ help: 'Cumulative count of actors that have been stopped.' },
).inc();
src/internal/ActorCell.ts:947-950
metricsOf(this.system).counter(
'actor_restarted_total', {},
{ help: 'Cumulative count of supervisor-driven actor restarts.' },
).inc();
src/internal/ActorCell.ts:967-971
metricsOf(this.system).counter(
'actor_mailbox_dropped_total',
{ class: cls, path: this.path.toString(), reason },
{ help: 'Cumulative count of user messages dropped by a bounded mailbox\'s overflow policy.' },
).inc();
src/internal/ActorCell.ts:987-990
metrics.counter(
'actor_messages_delivered_total', {},
{ help: 'Cumulative count of user messages delivered to actor onReceive.' },
).inc();
src/internal/ActorCell.ts:1044-1047
metrics.histogram(
'actor_message_handler_seconds', {},
{ help: 'Time spent inside actor onReceive handlers, seconds.' },
).observe(elapsedMs / 1000);
src/cluster/Cluster.ts:771-774
metricsOf(this.system).counter(
'cluster_gossip_rounds_total', {},
{ help: 'Cumulative count of gossip-push rounds initiated by this node.' },
).inc();
src/cluster/Cluster.ts:1102-1105
metricsOf(this.system).gauge(
'cluster_members_up', {},
{ help: 'Number of cluster members currently in `up` state.' },
).set(this.upMembers().length);
That is the whole set — actor_created_total, actor_terminated_total, actor_restarted_total, actor_mailbox_dropped_total, actor_messages_delivered_total, actor_message_handler_seconds, cluster_gossip_rounds_total, cluster_members_up. docs/src/content/docs/observability/metrics/stock-metrics.mdx lists exactly these eight and is correct; the operations section is the outlier.
Difference, docs → source. Six distinct names appear under docs/src/content/docs/operations/** that nothing in src/ emits.
docs/src/content/docs/operations/troubleshooting.mdx:69-72
```promql
rate(cluster_unreachable_duration_ms_count[5m]) > 0.1
# Frequent transitions
```
docs/src/content/docs/operations/troubleshooting.mdx:101-104
```promql
sharding_shards_hosted{type="entity-type"}
# Should equal numShards (total) across the cluster
```
docs/src/content/docs/operations/troubleshooting.mdx:148-150
```promql
histogram_quantile(0.99, persistence_recovery_duration_ms_bucket)
```
docs/src/content/docs/operations/troubleshooting.mdx:198-201
```promql
actor_mailbox_size{class=...}
# Find actors with persistently large queues
```
docs/src/content/docs/operations/tuning/dispatcher-tuning.mdx:141-155
Use the
[stock metric](/observability/metrics/stock-metrics/)
`actor_message_duration_ms`:
```
P50 ≈ work time
P99 - P50 ≈ dispatcher latency (queueing)
```
If P99 is far higher than P50 with little work-time variance,
**dispatcher tuning helps**.
The
`actor_mailbox_size` gauge under load shows whether actors are
keeping up. Persistently growing depth = either a slow handler
docs/src/content/docs/operations/upgrades/rolling-migration.md:108-110
- v1 events are being read by the new code path
(`migration_chain_upcast_total{from="1",to="2"}` metric, or a
log-line spot-check on `chain.read`).
mailbox-sizing.mdx compounds it — it presents actor_mailbox_size as a stock metric and then names both metrics in the prose under a third spelling that matches neither the invented one nor the real one:
docs/src/content/docs/operations/tuning/mailbox-sizing.mdx:131-144
Stock metrics
([Stock metrics](/observability/metrics/stock-metrics/))
expose mailbox depth:
```
actor_mailbox_size{class="Worker", path="..."}
actor_mailbox_dropped_total{class="Worker", path="...", reason="drop-head"}
```
Watch:
- **`mailbox_size`** — high values relative to capacity indicate
pressure.
- **`mailbox_dropped_total`** — non-zero with `drop-head` /
The section overview repeats the promise at the top:
docs/src/content/docs/operations/overview.mdx:129-132
<Aside type="caution" title="The second place is metrics">
Stock metrics give you actor message-rate, mailbox depth, cluster
member count, sharding region count — enough to identify
"something is off" before users notice.
Neither mailbox depth nor sharding region count is emitted by anything in src/.
Difference, source → docs. Of the eight metrics the framework actually emits, seven never appear anywhere under docs/src/content/docs/operations/**. The one exception is actor_mailbox_dropped_total at mailbox-sizing.mdx:137. The runbook and the instrumentation share one name out of eight.
Two adjacent sites belong in the same sweep. docs/src/content/docs/observability/management/http-endpoints.mdx:138-140 shows a sample GET /metrics response for actor_messages_processed_total{class="Worker",path="..."} — wrong name (the real one is actor_messages_delivered_total) and labels that family does not carry. By contrast docs/src/content/docs/operations/tuning/failure-detector.mdx:185 uses cluster_unreachable_total legitimately: it is a user-written sample that registers the counter itself, not a framework claim.
Every error above is mirrored verbatim in docs/src/content/docs/de/operations/.
Fix
Rewrite the operations diagnostics against the eight metrics that exist, and stop advertising ones that do not.
troubleshooting.mdx — replace all four PromQL blocks. Unreachable flapping is visible as a drop in cluster_members_up plus the cluster event stream, not as a cluster_unreachable_duration_ms histogram. Mailbox pressure is rate(actor_mailbox_dropped_total[5m]), not a depth gauge. Sharding coverage and persistence-recovery latency have no metric at all — say so and point at the management endpoints and the logs, rather than at a query that silently returns nothing.
dispatcher-tuning.mdx — actor_message_duration_ms is actor_message_handler_seconds (seconds, not milliseconds); delete the actor_mailbox_size paragraph.
mailbox-sizing.mdx — delete actor_mailbox_size; use the one real name consistently in both the code block and the bullets.
operations/overview.mdx — remove "mailbox depth" and "sharding region count" from the stock-metric promise.
rolling-migration.md — migration_chain_upcast_total does not exist; the log-line spot-check it already offers as the alternative is the only real option.
observability/management/http-endpoints.mdx — fix the sample scrape to a real name and real labels.
- Mirror every change into
docs/src/content/docs/de/.
The invented metrics are individually reasonable things to want. Where one is genuinely wanted it belongs in a feature issue — mailbox depth is #196 — not in a runbook as though it shipped.
Add a guard so the drift cannot recur: a test that collects metric-name literals from src/ and asserts every metric-shaped identifier in docs/** is either in that set or inside a block explicitly marked as user-registered. tests/unit/config/NoDeadConfigKeys.test.ts already does exactly this for HOCON keys. #470 proposes the general doc-drift CI guard (typecheck fenced samples, assert documented defaults) — a metric-name check is the same mechanism applied to a different vocabulary and belongs in the same harness.
Acceptance sketch
Verification status
Found in the ten-lens production-readiness review of 2026-08-05 (v0.13.0) and re-verified before filing: confirmed by reading. The emitted set was built by grepping every .counter( / .gauge( / .histogram( call site in src/ and reading the name argument at each — 8 sites, 8 names, all quoted above. The docs set was built with grep -rnoE "\b(actor|cluster|sharding|persistence|migration|mailbox)_[a-z0-9_]+" docs/src/content/docs/operations/. Both set differences are stated above and reproduce from those two greps alone.
Related: #196 asks for the mailbox-depth histogram these docs already describe as shipped; #658 and #745 concern the label set of the one stock metric the runbook does name correctly. #744 is the adjacent failure mode where GET /metrics answers 200 with an empty body — same 3am symptom of a query returning nothing, different cause. This issue is the docs-vs-code divergence only; it asks for no new metric.
Part of the production-readiness review batch — tracked in #913.
Where
docs/src/content/docs/operations/troubleshooting.mdx(all four PromQL blocks)docs/src/content/docs/operations/tuning/dispatcher-tuning.mdxdocs/src/content/docs/operations/tuning/mailbox-sizing.mdxdocs/src/content/docs/operations/overview.mdxdocs/src/content/docs/operations/upgrades/rolling-migration.mddocs/src/content/docs/observability/management/http-endpoints.mdx(sample scrape output)docs/src/content/docs/de/What is wrong
The operations section is the runbook an on-call engineer opens at 3am. All four of its PromQL blocks query a metric family that
src/never creates. Against a realactor-tsprocess every one of them returns an empty result — which in Grafana, and in an alert rule, is indistinguishable from "the system is healthy".The complete set of metric names emitted anywhere in
src/is eight. Every emission site, verbatim:That is the whole set —
actor_created_total,actor_terminated_total,actor_restarted_total,actor_mailbox_dropped_total,actor_messages_delivered_total,actor_message_handler_seconds,cluster_gossip_rounds_total,cluster_members_up.docs/src/content/docs/observability/metrics/stock-metrics.mdxlists exactly these eight and is correct; the operations section is the outlier.Difference, docs → source. Six distinct names appear under
docs/src/content/docs/operations/**that nothing insrc/emits.mailbox-sizing.mdxcompounds it — it presentsactor_mailbox_sizeas a stock metric and then names both metrics in the prose under a third spelling that matches neither the invented one nor the real one:The section overview repeats the promise at the top:
Neither mailbox depth nor sharding region count is emitted by anything in
src/.Difference, source → docs. Of the eight metrics the framework actually emits, seven never appear anywhere under
docs/src/content/docs/operations/**. The one exception isactor_mailbox_dropped_totalatmailbox-sizing.mdx:137. The runbook and the instrumentation share one name out of eight.Two adjacent sites belong in the same sweep.
docs/src/content/docs/observability/management/http-endpoints.mdx:138-140shows a sampleGET /metricsresponse foractor_messages_processed_total{class="Worker",path="..."}— wrong name (the real one isactor_messages_delivered_total) and labels that family does not carry. By contrastdocs/src/content/docs/operations/tuning/failure-detector.mdx:185usescluster_unreachable_totallegitimately: it is a user-written sample that registers the counter itself, not a framework claim.Every error above is mirrored verbatim in
docs/src/content/docs/de/operations/.Fix
Rewrite the operations diagnostics against the eight metrics that exist, and stop advertising ones that do not.
troubleshooting.mdx— replace all four PromQL blocks. Unreachable flapping is visible as a drop incluster_members_upplus the cluster event stream, not as acluster_unreachable_duration_mshistogram. Mailbox pressure israte(actor_mailbox_dropped_total[5m]), not a depth gauge. Sharding coverage and persistence-recovery latency have no metric at all — say so and point at the management endpoints and the logs, rather than at a query that silently returns nothing.dispatcher-tuning.mdx—actor_message_duration_msisactor_message_handler_seconds(seconds, not milliseconds); delete theactor_mailbox_sizeparagraph.mailbox-sizing.mdx— deleteactor_mailbox_size; use the one real name consistently in both the code block and the bullets.operations/overview.mdx— remove "mailbox depth" and "sharding region count" from the stock-metric promise.rolling-migration.md—migration_chain_upcast_totaldoes not exist; the log-line spot-check it already offers as the alternative is the only real option.observability/management/http-endpoints.mdx— fix the sample scrape to a real name and real labels.docs/src/content/docs/de/.The invented metrics are individually reasonable things to want. Where one is genuinely wanted it belongs in a feature issue — mailbox depth is #196 — not in a runbook as though it shipped.
Add a guard so the drift cannot recur: a test that collects metric-name literals from
src/and asserts every metric-shaped identifier indocs/**is either in that set or inside a block explicitly marked as user-registered.tests/unit/config/NoDeadConfigKeys.test.tsalready does exactly this for HOCON keys. #470 proposes the general doc-drift CI guard (typecheck fenced samples, assert documented defaults) — a metric-name check is the same mechanism applied to a different vocabulary and belongs in the same harness.Acceptance sketch
docs/src/content/docs/operations/**names a metric that a grep ofsrc/finds as a string literal.actor_mailbox_size,actor_message_duration_ms,sharding_shards_hosted,persistence_recovery_duration_ms,cluster_unreachable_duration_msandmigration_chain_upcast_totalappear nowhere indocs/.operations/overview.mdxpromises only metrics that exist.observability/management/http-endpoints.mdxshows a scrape sample with a real name and real labels.docs/that nosrc/file emits.Verification status
Found in the ten-lens production-readiness review of 2026-08-05 (
v0.13.0) and re-verified before filing: confirmed by reading. The emitted set was built by grepping every.counter(/.gauge(/.histogram(call site insrc/and reading the name argument at each — 8 sites, 8 names, all quoted above. The docs set was built withgrep -rnoE "\b(actor|cluster|sharding|persistence|migration|mailbox)_[a-z0-9_]+" docs/src/content/docs/operations/. Both set differences are stated above and reproduce from those two greps alone.Related: #196 asks for the mailbox-depth histogram these docs already describe as shipped; #658 and #745 concern the label set of the one stock metric the runbook does name correctly. #744 is the adjacent failure mode where
GET /metricsanswers 200 with an empty body — same 3am symptom of a query returning nothing, different cause. This issue is the docs-vs-code divergence only; it asks for no new metric.Part of the production-readiness review batch — tracked in #913.