Skip to content

fix: escalate supervisor-monitor failures instead of hanging - #36

Merged
dbrattli merged 1 commit into
mainfrom
fix/supervisor-escalate-monitor-failures
Aug 1, 2026
Merged

fix: escalate supervisor-monitor failures instead of hanging#36
dbrattli merged 1 commit into
mainfrom
fix/supervisor-escalate-monitor-failures

Conversation

@dbrattli

@dbrattli dbrattli commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator

Closes #28. Second of the issues recorded from the #25 review.

The bug

The monitor in flatMapActorSupervised is the top of its supervision tree — spawned with Actor.spawn, not spawnLinked, so nothing watches it in turn — and its loop body was bare. Three things in it can throw:

  • decide ex — a user-supplied exn -> Directive callback
  • aobv.OnErrorAsync ex on the Escalate path
  • unbox<'TSource> msg / child.Post on the forward path

Any of them killed the loop outright. The stream then hung rather than failed: upstream kept calling OnNextAsync, the posts landed in a mailbox nobody read, and the observer saw neither OnCompleted nor OnError. On .NET the exception surfaced only on MailboxProcessor's unobserved Error event; on BEAM not at all.

The fix

Guard the body, report downstream, stop looping:

let! alive =
    actor {
        try
            // ... existing body ...
            return true
        with ex ->
            do! escalate ex
            return false
    }

if alive then
    return! loop ()

Two deliberate choices:

  • Stop after escalating. Once OnError has gone downstream the Rx grammar is closed, so there is nothing left to forward. The composite disposable still tears down the source and inner subscriptions.
  • escalate swallows secondary failures. An exception thrown while reporting would reproduce the exact bug being fixed, so the reporting path cannot be allowed to throw.
let escalate (ex: exn) =
    async {
        try
            do! aobv.OnErrorAsync ex
        with _ ->
            ()
    }

Why the guard is a nested actor { } and not async { }

It keeps child <- ... executing inside the actor process. On BEAM a captured let mutable is lowered to the process dictionary, so moving the assignment into a block that might run elsewhere would silently break restarts. Confirmed in the generated Erlang that it did not:

fable_actor_actor:actor_builder_try_with_...   % actor builder, not async
fable_actor_actor:actor_1_post_2_b595(erlang:get(Child_ref), Msg),
erlang:put(Child_ref, fable_actor_actor:actor_spawn_linked(Inbox, ...))

actor_trap_exits() from #32 is still emitted ahead of it.

Test

Asserts the decider's own exception is what reaches downstream, not the child's:

xs
|> Reactive.flatMapActorSupervised
    (fun _ -> raise deciderError)      // decider throws inside the monitor loop
    (fun emit inbox -> ... failwith "child boom" ...)
...
do! obv.WaitUntil(List.exists isOnError)
assertThat reported (isEqualTo (Some "decider boom"))

Verified red-green. Without the src fix it fails on Quill's timeout at 5003ms — which is the silent hang itself, not an assertion failure — and passes in 45ms with it.

Verification

  • just test87 passed (87)
  • dotnet fable src --lang <beam|javascript|python> → all three exit 0 (and now gated by ci: gate the Fable compile targets #35)
  • dotnet fantomas --check clean on all three files

Adds isOnError to Tests.Utils, alongside the isOnNext/isOnCompleted added in #33.

🤖 Generated with Claude Code

Closes #28.

The monitor in `flatMapActorSupervised` is the top of its supervision tree —
`Actor.spawn`, not `spawnLinked`, so nothing watches it in turn — and its loop
body was bare. A throw from the user's `decide` callback, from `aobv.OnErrorAsync`
on the Escalate path, or from the `unbox`/`Post` forward killed the loop outright.

The stream then hung rather than failed: upstream kept calling `OnNextAsync`, the
posts landed in a mailbox nobody read, and the observer saw neither OnCompleted
nor OnError. On .NET the exception surfaced only on MailboxProcessor's unobserved
Error event; on BEAM not at all.

Guard the body, report the failure downstream, and stop looping — after an
OnError the Rx grammar is closed, so there is nothing left to forward. The
escalation path itself swallows secondary failures, since an exception thrown
while reporting would reproduce the exact bug being fixed.

The guard is a nested `actor { }` rather than `async { }` on purpose: it keeps
`child <- ...` inside the actor process. Confirmed in the generated Erlang — the
loop uses the actor builder's try_with and `child` stays process-dictionary
backed via erlang:get/put.

Test asserts the decider's own exception reaches downstream. Verified red-green:
without the fix it fails on Quill's 5003ms timeout — the silent hang itself —
and passes in 45ms with it.

Adds `isOnError` to Tests.Utils.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@dbrattli
dbrattli merged commit 2c3fbd6 into main Aug 1, 2026
5 checks passed
@dbrattli
dbrattli deleted the fix/supervisor-escalate-monitor-failures branch August 1, 2026 10:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

flatMapActorSupervised: monitor crash silently kills the subscription

1 participant