Skip to content

[Bug] context.watch() is a silent no-op for a remote ref, so cross-node death watch never delivers Terminated despite the JSDoc promising it unconditionally #918

Description

@pathosDev

Problem

context.watch(remoteRef) is a silent no-op. Death watch across a cluster boundary does not exist, and nothing says so — ActorContext.watch's JSDoc promises unconditionally that a Terminated message is sent when the watched actor stops.

The consequence is that every supervision pattern built on death watch degrades silently the moment the watched actor is on another node: a watcher waiting for Terminated to trigger failover, replace a worker, or release a resource simply waits forever. Nothing logs, nothing throws, and it works perfectly in a single-node test.

Evidence

watch() registers the watcher only for a local ref:

src/internal/ActorCell.ts:494-503
  watch(ref: ActorRef): ActorRef {
    const key = ref.path.toString();
    if (this._watching.has(key)) return ref;
    this._watching.set(key, ref);
    if (ref instanceof LocalActorRef) {
      ref.getCell()._addWatcher(this.self);
    }
    return ref;
  }

For a RemoteActorRef the _watching map is updated and nothing else happens.

The system command that would carry a remote death notice has no producer anywhere in the repository. A repo-wide grep for watchNotify outside its own type declaration returns exactly two hits — a passing mention in a comment, and the handler:

src/ActorContext.ts:210:   * supervision, watchNotify) are NOT throttled — they always run
src/internal/ActorCell.ts:766:      .with({ kind: 'watchNotify' }, (signal) => this.onWatchNotify(signal))

src/cluster/RemoteActorRef.ts contains zero occurrences of watch or Terminated (grep count: 0), so there is no remote side either.

The contract as documented:

src/ActorContext.ts:126
   * Start death-watching an actor.  A Terminated message is sent when it stops.

Proposal

Two acceptable outcomes; the current one is neither.

  1. Implement it. The receiving node registers a remote watcher, the owning node emits watchNotify over the wire when the watched actor terminates, and the watcher's node also synthesises Terminated when the owning node leaves or is downed — the second half is what makes it useful, since node loss is the case death watch exists for.
  2. Refuse it loudly. watch() throws on a non-local ref with a message naming the limitation, and the JSDoc states the constraint.

A silent no-op behind an unconditional promise is the worst of the three, because it fails only in the environment the feature is for.

Acceptance sketch

  • context.watch(remoteRef) either delivers Terminated when the remote actor stops, or throws with an actionable message.
  • If implemented: a watcher receives Terminated when the owning node is downed, not only when the actor stops cleanly.
  • ActorContext.watch's JSDoc matches whichever behaviour ships.
  • A multi-node test covers the case — no test currently exercises watch across a node boundary.

Verification status

Found in the ten-lens production-readiness review of 2026-08-05 (v0.13.0) and re-verified before filing: confirmed by reading the cited lines and by repo-wide greps for watchNotify producers and for watch/Terminated in RemoteActorRef.ts. Not reproduced at runtime, because doing so requires a two-node cluster and the absence of a producer is already decisive.

Part of the production-readiness review batch — tracked in #913.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpriority: highTop priority — high impact, plan nextproduction-goalBlocks or defines the path to production readiness

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions