Detect silent faults in the ROS 2 graph

[bburda]

Summary

Some failures leave the graph looking healthy. Every node is up, nothing logs an error, /health is green, and the robot still does not work.

Examples we keep hitting:

• A publisher offers BEST_EFFORT and a subscriber requests RELIABLE. DDS never matches the pair, so the topic exists, both nodes are alive, and no message is ever delivered.
• A remap typo. One node publishes /scan_raw, another subscribes /scan_row. Both sides are healthy and neither is talking to the other.
• A node dies. The thing that would have reported it is the thing that died.
• A TF broadcaster stops publishing while its process stays alive. Localization and planning break; the graph shows nothing wrong.
• A parameter drifts away from the value the robot was commissioned with.
• A node the operator expects to be active is stuck in unconfigured after a failed bringup.
• A pipeline stage goes silently late, so data still flows but too old to use.

None of these produce a fault today. An operator finds them by noticing the robot behaves badly and then reading topics by hand.

Proposed solution

A gateway plugin that watches the ROS 2 graph and raises a fault for each of these classes, under frozen GRAPH_* fault codes so downstream consumers can match on them.

Two things it has to get right to be usable:

• No bringup noise. A joining node, a topic still being discovered, and a lifecycle node mid-transition all look like failure for the first few ticks. Raises need to be held until the graph settles.
• No silent healing. Once a fault is raised it must not disappear because the evidence went away. A dead node that stops being visible is still dead. Suppression should be opt-in and explicit, never a side effect.

Detectors that need an operator contract (expected parameter values, required-active nodes, per-topic timing budgets) should watch nothing until configured, so the plugin cannot false-positive on a graph nobody set up.

---

Breakdown and order

One change per fault class, each with its own issue. They merge one at a time: a PR here that targets another PR's branch triggers no workflow at all, because ci.yml, quality.yml and docs.yml all filter on branches: [main]. So each one goes in before the next opens.

#	Change	Issue	Depends on
1	Plugin scaffolding + qos_mismatch	#572	-
2	orphan	#577	1
3	param_drift	#579	1
4	lifecycle_expectation		1
5	Suppression framework + node_death		1
6	tf_stale		5
7	timing_budget		5

Why the dependencies fall that way.

Everything needs 1: the detector interface, the registry, the config plumbing, the reliability gate and the aggregated-fault helper all live there.

Beyond that the detectors split by whether they need the suppression framework. orphan, param_drift and lifecycle_expectation do not, so 2, 3 and 4 are independent of each other and can be reordered freely. node_death, tf_stale and timing_budget all do.

Suppression is not a fault class, so it gets no issue of its own. It lands with node_death, its first consumer, which makes 5 the largest change in the set. 6 and 7 then only add their detector.

One carried-over piece: the end-to-end proof that nested per-detector config actually reaches a detector is written against param_drift's expect rule, so it travels with 3 rather than 1.