fix: server failed to acknowledge more than 5 heartbeats - connection is zombie

[gretmn102]

Last seen

The last time such an error was observed was in 2022. I may have accidentally fixed it at some point (see possible causes). Perhaps we should close the issue.

Reproduction

???

Possible causes {#posible-causes}

This can happen because of an exception in MailboxProccessor: when .PostAndReply is called, it waits for a response. If an exception occurs in MailboxProccessor before that, .PostAndReply will continue to wait indefinitely.

The following code can be used to prove this:

module AsyncLearning =
    open System.Threading.Tasks

    let awaiti (t: Task) =
        t.GetAwaiter().GetResult()

    type Msg =
        | Inc of AsyncReplyChannel<int>
        | GetCounter of AsyncReplyChannel<int>

    let reduce msg state =
        match msg with
        | Inc ->
            let counter = state + 1
            if counter > 1 then
                System.Threading.Tasks.Task.Run(fun x ->
                    failwithf "counter > 1"
                )
                |> awaiti

            counter

        | GetCounter r ->
            r.Reply state
            state

    let counter =
        let initState = 0

        MailboxProcessor.Start(fun mail ->
            let rec loop state =
                async {
                    let! msg = mail.Receive()
                    let state =
                        try
                            reduce msg state
                        with e ->
                            printfn "%A" e
                            state

                    return! loop state
                }
            loop initState
        )

    counter.PostAndReply (fun r -> Inc r)

    ()