Singleton partition cannot self-recover when RX is alive but MAC TX stalls

[byran-pro]

ot-br-posix on OpenWrt-based product + RCP, BR role during issue is Router/Leader.
GSDK_4.4.6 https://github.com/SiliconLabs/gecko_sdk/tree/gsdk_4.4
The same issue with openthread/ot-br-posix#2473

In some partition events, the Border Router enters singleton mode and continues to receive Thread traffic, but local MAC data TX stops making forward progress. In this state, partition recovery does not complete without restarting otbr-agent.

Failure signature observed repeatedly:

singleton remains true
MAC RxTotal continues to increase
MAC TxData (and often TxTotal) does not increase
BR remains stuck until otbr-agent restart
1 OTBR Gateway + 15 TME(Thread Mesh Extender) + 85 End devices.
Keep data transmission stream about 2 weeks.
A representative timeline from one reproduced run:
(Ref log full_log.log and full_log_2.log)

18:34:10 -> singleton detected (watchdog log: partition-guard singleton begin)
18:39:11 -> condition persisted for about 300 seconds with RX alive and no TxData progress
18:39:11 -> restart triggered as mitigation, after which topology rebuild events resumed
The core issue is the non-self-recovering state before restart.

full_log.log
full_log_2.log

[jwhui]

Failing to service the message queue (often observed as the message queue filling up until it hits its limit) typically indicates an issue within the platform-specific radio driver.

In OpenThread, the MeshForwarder manages outbound message queues. When a message is ready to be sent, the stack submits it to the platform layer via otPlatRadioTransmit(). The core stack then waits for the platform to signal that the transmission has completed (successfully or otherwise) through the otPlatRadioTxDone() callback.

If the radio driver does not call otPlatRadioTxDone(), the stack will not dequeue the current message and will not attempt to send the next one in the queue. This causes the queue to grow until it hits limits (e.g., OPENTHREAD_CONFIG_MAX_FRAMES_IN_DIRECT_TX_QUEUE), at which point you may see logs such as:

• Dropping (dir queue full)
• Out-of-buffer errors (kErrorNoBufs)

Troubleshooting Suggestions:

1. Callback Verification: Ensure that your radio driver calls otPlatRadioTxDone() for every call to otPlatRadioTransmit(). This includes cases where the transmission failed (e.g., due to Channel Access failure or no ACK received).
2. Driver State Machine: Audit the radio driver's state machine. If the driver gets "stuck" in a transmitting state and never returns to a state where it can process the next request, the queue will back up.
3. Interrupt Handling: Check if transmission completion interrupts are being correctly handled and that the resulting callback is scheduled correctly.
4. Flow Control: If you are using a Radio Co-Processor (RCP) architecture, ensure that the Spinel/HDLC interface between the Host and RCP is functioning correctly and not dropping completion events.