Proposal: URMA runtime recovery and business QoS configuration

[Spicy-cream]

URMA applications need recovery after an established connection fails and independent QoS settings for different business channels. I would like to discuss these two related changes:

• Runtime recovery and TCP fallback, submitted to Apache brpc.
• Business QoS configuration, submitted to Apache brpc and based on the recovery change above. Development PR.

Runtime recovery

Connection-setup fallback alone does not guide subsequent connections after a runtime failure. Repeated connections can continue selecting the same failing URMA peer.

The proposal adds process-level LEGACY, AUTO and TCP_ONLY modes. LEGACY remains the default. AUTO places a failed peer into a configurable cooldown, defaulting to five seconds. New connections use TCP during cooldown and can select URMA again afterward. Mode changes affect new connections; established single/pooled TCP connections can continue serving RPCs.

Failures are reported once per connection, and epochs filter reports from connections created before a policy change. Completion, JFC and WR-posting failures feed recovery decisions. Access errors are counted separately, while send-queue and receive-credit backpressure retain EAGAIN behavior. Recovery status and admission counters are exposed through bvars.

Business QoS

The proposal adds outgoing QoS settings to ChannelOptions and ServerOptions, allowing request and response traffic to select SL and priority independently.

SL is resolved through the selected device's CTP/RTP priority-to-SL mapping. A requested SL must have a compatible priority on the selected path. Explicit SL/priority pairs must match that mapping. Unsupported, conflicting and out-of-range configurations fail during Channel initialization or Server startup, with supported pairs in the diagnostic message.

QoS participates in connection reuse and propagates to accepted, pooled and short sockets, as well as rebuilt Jettys. Connection diagnostics expose requested and effective values.

Validation

URMA RM/RTP hardware checks passed runtime mode switching, v2/v3 handshakes, short/single/pooled connections and 64 KiB/1 MiB payload validation. SEND-post fault injection exercised TCP cooldown and subsequent RPC recovery; new short connections resumed URMA after cooldown.

QoS hardware checks passed 12 event/polling, v2/v3 and configuration combinations, plus six client/server configuration-rejection cases. A live server connection confirmed the effective priority and SL.

Mock regressions cover CTP/RTP selection and server-SEND failure recovery. The recovery and QoS branches pass 15 and 21 regression groups respectively. New tests are added separately from existing community UT sources.

Feedback requested

1. Are process-level mode selection and per-peer cooldown suitable recovery controls for URMA applications?
2. Does ChannelOptions/ServerOptions provide the right scope for request and response QoS configuration?
3. Should an unsupported SL fail initialization, as proposed, or should the API expose an explicit fallback policy?