Skip to content

Performance and Design Notes

mchristegh edited this page Jul 14, 2026 · 1 revision

The honest page: what recording costs, what the recorder cannot do, and why it works the way it does. The design-rationale section exists so future changes (ours or a fork's) don't accidentally "fix" something that is load-bearing.

The cost model

Every captured message is cloned and encoded — circular-safe, type-aware, redaction applied. For the inline recorder that cost sits on one wire you chose; for a tap it applies to every in-scope emission, and the scope check itself runs on every emission in the runtime (a cached set lookup — negligible; it's the capture that costs).

For typical home-automation and integration flows this is unmeasurable. The place to think is a genuinely chatty source — a high-frequency MQTT firehose, a polling loop at 10Hz. The levers, in order of effectiveness:

  1. Summary capture mode — stores type, size, preview, and keys instead of the encoded message. An order of magnitude lighter, and for "what was the traffic shape before the failure" it's often all you need.
  2. Exclude or mute the loudest sources. Exclusion if permanent, mute if operational. A muted source still costs the scope check, not the capture.
  3. Modest capacity and Max bytes. Memory is roughly capacity × typical record size per recorder; a 100-record buffer of summary records is a few tens of kilobytes.
  4. Multiple taps multiply. Two taps with overlapping scopes each clone and buffer independently — deliberate (each recording is self-contained), but memory and CPU double. Prefer one tap with a wider scope over several overlapping ones.

Persistence and flash storage. Captures persist via write-behind (every 20 captures or 5 seconds, whichever first; state-changing commands write immediately). On flash-based hosts (Raspberry Pi SD cards, Home Assistant Yellow eMMC), a persistent recorder on sustained traffic means regular small writes. Guidance: enable persistence where crash-survival genuinely matters, not by default everywhere; summary mode shrinks the writes too. This is a diagnostic tool, not a message historian — for the latter, use a database.

Known limitations

Documented rather than hidden:

  • Hooks see messages between nodes. Not inside function logic; not messages a node swallows (visible only as cadence gaps); not anything during a hard crash — persistence is the crash story, hooks are the running story.
  • Auto-error dump needs honest nodes. It rides node.error(err, msg); most nodes report properly, some older contrib nodes swallow errors silently.
  • Inline sources are the recorder itself. Node-RED doesn't pass sender identity on wires; only the tap's hooks attribute true sources.
  • Tap scope cache and moved nodes. A node moved between flows keeps its cached in/out verdict until the tap redeploys. Additions self-heal; moves don't.
  • Subflow internals belong to the subflow's internal flow, so flow-scoping a tab doesn't reach inside instances on it; scope by type instead.
  • deltaMs uses wall-clock time, so a host clock adjustment (NTP step) can produce one odd delta.
  • No editor-side viewer. Recordings are read via reports and files, not a sidebar UI — a deliberate v1 scope choice (see Roadmap).

Design rationale

Passthrough is sacred. The inline recorder forwards the original object — same reference, untouched — and no capture failure may block it. Rationale: an observability tool that can break the thing it observes is worse than no tool. This is also why capture errors surface as events instead of throwing, why the tap wraps every hook callback defensively, and why pausing stops the tape, never the flow.

Nothing invisible. Paused messages are counted; suppressed auto-dumps emit events; mutes mark the tape and count what they hide; commands acknowledge; partial file writes report per-file. Rationale: a diagnostic tool that silently drops information poisons the diagnosis. Every suppression mechanism in the package accounts for itself.

Recorders never record recorders — hard-coded, not configurable. Rationale: a tap capturing its own incident emissions is a feedback loop; making the exclusion unconditional removes an entire failure class instead of documenting a footgun.

Evidence is never destroyed. Dumps don't clear the tape; file collisions suffix instead of overwrite; retention prunes report+JSON pairs and only files matching the node's own template. Rationale: this is a black box — the moment you most need the data is the moment after something went wrong, which is the worst moment to discover an overwrite.

Sequence numbers are monotonic forever — across wraps, clear, and restarts. Rationale: records from different dumps, files, and sessions must be globally orderable; a reused seq would make two records ambiguous exactly when correlating evidence.

Commands are out-of-band for the inline recorder. Its input carries traffic; a wire carrying arbitrary payloads must never be ambiguous about what's a command (a legitimate "clear" payload must not wipe the tape). The control node exists so command addressing is structural, not heuristic. The tap accepts payload commands only because its input has no other purpose.

Mutes are ephemeral; exclusions are permanent. Configuration defines what could be recorded (stable, interpretable); commands adjust operation (reset on deploy). A mute surviving a restart unnoticed is a data-loss trap; the marks it leaves on the tape preserve history without persisting the state.

Manual dumps bypass the cooldown. The cooldown exists to stop machines (error loops), not people. A human asking for the tape gets the tape.

The report node is stateless and JSON is raw. Formatting is a pure function of the incident, so many report nodes can consume one output without interference; the JSON companion is the untouched incident rather than an enriched export, because analysts want source data, not precomputed opinions — and it keeps the internal report model free to evolve.

Zero runtime dependencies. Everything is Node stdlib. Rationale: a diagnostic tool should be the most boring thing in your palette to install, audit, and keep working — and integrations (databases, Home Assistant) are better served by wiring into their mature ecosystem nodes than by us bundling drivers.

Clone this wiki locally