"Knock-come" solving a problem but my tokio implementation seems to reintroduce it

[Aclassifier]

I have this "knock-come" pattern that I introduced in 2009. I wrote about it then, and there has been added three new implementations recently. See the blog note at https://www.teigfam.net/oyvind/home/technology/009-the-knock-come-deadlock-free-pattern/ (no ads or money involved, just a hobby).

Since I am new at Rust and tokio I have had Google AI help me make this implementation, based on an implementation in the XMOS XCORE xC language. The Rust code and log is at https://github.com/Aclassifier/rust_test_knock_come.

The problem with this Rust code is described in the note. Basically, in order to avoid a deadlock, it is that I seem to have to use a try_send from the master task to the slave task on a random timeout when the master task wants to send data spontaneously to the slave task. With this almost "busy poll send" the architectural deadlock problem that knock-come solves is in some way reintroduced to avoid a deadlock (again) with a try_send.

AI and I tried several other ways, but the deadlock appeared for all except the solution with the try_send. But I assume that tokio experts might have a second view on this. Is there any other way to implement knock-come in Rust and tokio?

Observe that I am not asking about whether the top level architectural deadlock problem might be solved differently. It may! I am asking about the knock-come pattern and Rust and tokio and flume.

[MentalOfCrow]

I read the current v0.920 implementation and the three logs. There are two separate questions hidden in the behavior:

1. Is it acceptable to skip a spontaneous value while the slave is busy with the handshake?
2. If not, where should backpressure live without preventing the master from servicing a Knock?

First, the current try_send is not busy polling. It is called once per random timer expiry and returns once. With a zero-capacity Flume channel it means “perform this spontaneous rendezvous only if the receiver is already ready”; your /eN counter is the number of spontaneous events deliberately not delivered. If that is the intended semantic, try_send is the correct non-blocking guard and there is no architectural deadlock in it.

The deadlock risk appears when try_send is changed to:

ch_come_or_sdata_tx.send_async(spontaneous_data).await

The master can then suspend while trying to send SpontaneousData on the same zero-capacity channel that must later carry the higher-priority Come. While suspended inside that selected timer branch, the master is not selecting ch_knock_rx. In other words, a selected branch contains another rendezvous whose completion depends on protocol progress handled by the outer loop. biased; changes which ready branch wins; it cannot break that wait cycle.

I would not multiplex Come and spontaneous traffic onto one rendezvous channel. They have different semantics and priority.

If only the latest spontaneous value matters

Use a Tokio watch channel for master-to-slave spontaneous state, and keep Come as a separate control path. watch retains one latest value and its send does not wait for the receiver, which exactly replaces the timer + zero-capacity try_send behavior without an error/drop counter:

let (spontaneous_tx, spontaneous_rx) =
    tokio::sync::watch::channel::<ExchangedDataT>(0);

// Master timer branch: never waits for the slave.
data_from_master += DATA_FIRST_AND_INC;
spontaneous_tx.send_replace(data_from_master);

The slave includes spontaneous_rx.changed() in its top-level select! and reads with borrow_and_update(). While waiting for Come, it can continue selecting both the control receiver and changed(). Multiple updates that happen faster than the slave can process are intentionally coalesced to the latest value.

If every spontaneous value must be delivered

Use a separate bounded tokio::mpsc data channel and drive its capacity from the same top-level state machine. Do not .await an ordinary send inside the timer branch. Store the pending message and select on Sender::reserve():

let mut pending_spontaneous: Option<ExchangedDataT> = None;

loop {
    tokio::select! {
        knock = knock_rx.recv() => {
            // Advance the Knock/Come state machine.
            handle_knock(knock).await;
        }

        _ = &mut local_timer, if pending_spontaneous.is_none() => {
            pending_spontaneous = Some(data_from_master);
            data_from_master += DATA_FIRST_AND_INC;
            reset_timer(&mut local_timer);
        }

        permit = spontaneous_tx.reserve(), if pending_spontaneous.is_some() => {
            match permit {
                Ok(permit) => permit.send(pending_spontaneous.take().unwrap()),
                Err(_) => break,
            }
        }
    }
}

reserve() waits for capacity as a selectable event, but it does not trap the actor inside the timer branch: the Knock receive remains live. The returned permit makes the actual send synchronous. This is also the Tokio-documented way to avoid losing a message when a send future is cancelled by another select! branch.

For strict priority, keep the control path separate and place it first under biased;, but only after establishing that every branch is cancellation-safe. Priority can prevent starvation; it is not a substitute for breaking channel dependency cycles.

So my recommendation for this particular demo is:

• keep ch_knock and the Knock/Come/SlaveData state machine as the protocol path;
• move SpontaneousData to its own channel;
• choose watch if it represents current state, or bounded mpsc + reserve() if every sequence value is required;
• avoid any zero-capacity send_async().await inside a selected branch when that actor must keep servicing another channel to complete the protocol.

References:

• Tokio watch (last-value channel): https://docs.rs/tokio/latest/tokio/sync/watch/
• Tokio mpsc::Sender::reserve and cancellation behavior: https://docs.rs/tokio/latest/tokio/sync/mpsc/struct.Sender.html#method.reserve
• tokio::select! lifecycle/fairness/cancellation notes: https://docs.rs/tokio/latest/tokio/macro.select.html
• Flume confirms that a zero-capacity channel is always full and send_async waits: https://docs.rs/flume/latest/flume/struct.Sender.html

If this distinction between optional state updates and lossless queued events resolves the design question, please mark it as the accepted answer so future readers can find the single-state-machine approach.

[Aclassifier]

Dear @MentalOfCrow. Thank you for this long and it looks like a very interesting answer! I will study it thoroughly over the next days and come back with my comments. I have felt somewhat stuck some days now, so this was refreshing! I'll be back. (None of the three logs of v0.920 seems to indicate any deadlock)

[Aclassifier]

I tried to answer inline, but didn't succeed. Maybe it's just as ok to have my comments loosely connected. However, I have added a PDF that may be more readable: 2026 07 18 Response from MentalOfCrow.pdf.

It is not acceptable to lose a spontaneous message from master to slave at any time. Skip, maybe, if the master knows it's not received and that it could send it again. This "sending" again is what I meant by "almost "busy poll send"".

Since it's not really the intended semantics, any use of try_send would only be a second choice, if there are alternatives. And I do understand that there is no busy poll per Tokio. It's the usage that then creates this rather not wanted almost busy poll send. Like try_send until sent. For v0.920 this is MasterTrySendSlaveSelect (meaning MasterTrySendSlave-Single-Select). It does not seem to deadlock. (Knowing that the absence over an hour or two of is no proof of deadlock freedom. But I trust that the Rust runtime or Tokio code itself do not have any error that wouldn't cause such a deadlock).

This would be my code now for (1) MasterSendSlaveNestedSelect (meaning Master-send_async(spontaneous data).await-SlaveNestedSelect) or (2) MasterForceSendSlaveSelect (meaning Master-send_async(spontaneous data).await-Slave-Single-Select)). When I run this code, none of the modes seem to cause any deadlock. The master does send_async(spontaneous data).await and would never block on it since the 1-capacity knock channel should allow that the slave's send_async(no data, meaning knock).await would really go on (to accept the input from the master in the next select loop) since it would never try to send more than once (since provided SlaveSentDataNowReady).

I guess that any message that carries some protocol would cause the semantics to change. I did notice that Claude suggested two channels for this, but I wanted explicitly to do it with one channel. But I agree that splitting up over two channels also has some understandability value. I am used to (starting with occam in the early nineties) that a chan is precious, and that most often two tasks would have one channel between them for messages in one concrete direction.

I didn't know about the watch channel. Interesting indeed. However, since I come from safety critical, it's in my bones never to accept messages to become lost. Or, I'd like to have control over which messages may be allowed to be discarded, like if there is no use in sending an older message on if a newer carries higher value. Like I didn't get rid of this fire alarm prewarning, but now I have got a fire alarm proper which I must send instead. (Maybe the prewarning also has value, needed for log and fire development analysing later on, but right now the proper alarm is more urgent). I once suggested the XCHAN that allowed older messages to de discarded, in a paper (how do I get these inline?):

https://www.teigfam.net/oyvind/home/technology/250-xchans-notes-on-a-new-channel-type/

https://www.teigfam.net/oyvind/pub/CPA2012/paper.pdf.

It's not the medium (the channel or pipe) that "loses" the message, it's the application.

MPSC = Multi-Producer, Single-Consumer = many-to-one channel. This is very interesting. But capacity 0 is not allowed, so I'd assume that this solves some other problem than my strict knock-come with exactly those channels and those capacities. Assuming that you are onto something, see next comment.

..next comment: isn't master recv_async() and &mut local_timer in the select safe? I thought so. You say that "This is also the Tokio-documented way to avoid losing a message when a send future is cancelled by another select". Does this mean that even if the two guards in the master select are safe then it's the send_async().await that may lose the sending because, even if it's not the guard but started inside a guard it still may be torn down by the knock trying to get through, and that a permit.send would solve this? I don't have the semantics here on board yet!

I am not used to thinking that a select (XC, Golang, or ALT in occam and in several runtimes I have written or [ ] or |~| in CSP) has any hidden (for me at least) features like they have their own lives, it seems to. Or am I blindly wrong? Is the Tokio select on Rust something that behaves "like if a select had been part of the Rust language"?

Thank you for your work! I guess that by me being stringent that it should be the two-task three channel cap 1, 0, 0 knock-come that I wanted to see in Rust (and learn from it), then by getting these suggestions for another kind of implementation, I should at least learn from! But then doubting my own reasoning there, you still have a means to solve the problem when two "synchronous" tasks would spontaneously want to tell something to the other. Maybe I should try to make a TaskSemantics of MasterMpscReserveSlaveSelect (or MasterMpscReserveSlaveNestedSelect)??

Adding: is is correct to conclude that none of the TaskSemantics in v0.920 should cause any deadlock or data to be lost?

[Aclassifier]

About the question in the last line above. It looks like for MasterForceSendSlaveSelect it does not deadlock.

However I think that according to the reasoning it should deadlock since ch_come_or_sdata_tx.send_async(spontaneous_data).await in master into a single slave select.

Should it have stumbled on the deadlock immediately (which it doesn't look like) or might the deadlock only happen very seldom? The only means I have to see the deadlock is to watch in the log, if nothing happens there (within 60 secs.) then there is a deadlock, I have concluded. Doing MS_USE_CONSTANT_TIMEOUT true or false does not seem to make any difference. Are the timer trigging by the master and slave too long and too low probablility to never hit each other kind of "in my life"?

Is there a smart way to make it deadlock immediately? Provided it's "supposed to" deadlock. And, to have it detect it itself and log the deadlock?