[Enhancement]: Expose safe typed asynchronous callbacks in the Rust native-plugin SDK

[afourniernv]

Affected area

Rust core runtime; Plugins; Middleware or guardrails.

Problem or opportunity

Relay's native ABI already supports completion-based asynchronous middleware through ABI v3, including cancellation-aware buffered and streaming operations. The high-level Rust plugin SDK exposes typed synchronous callbacks only.

A native plugin that performs network I/O through the safe typed API must therefore block inside the callback or implement the raw ABI itself. In Switchyard PR #270, the safe typed implementation waits with Receiver::recv() and recv_blocking(). Because Relay invokes those callbacks on normal Tokio workers, enough concurrent calls can occupy the entire worker pool.

A controlled comparison using the same Switchyard implementation before and after its typed-SDK refactor showed:

• Four routed calls on a four-worker Relay runtime delayed unrelated managed work by about 504 ms when the provider took 500 ms.
• The completion-based async implementation kept unrelated work near 1.45 ms even with all eight workers matched by eight routed calls.
• Caller cancellation returned promptly in both versions, but only the completion-based implementation closed the upstream buffered and streaming requests. The synchronous typed implementation left them running until response or timeout.

Increasing TOKIO_WORKER_THREADS only moves the saturation threshold.

Proposed enhancement

Add a safe typed wrapper over the existing completion-based native ABI for buffered and streaming middleware.

The API should let plugin code schedule asynchronous work and settle a host-owned completion/stream handle without passing Rust futures, trait objects, or allocator-owned values across the C ABI. It should preserve cancellation propagation and expose bounded stream push/backpressure behavior through safe Rust types.

Runtime contract and binding impact

This is a Rust native-plugin SDK enhancement. The underlying runtime and C ABI behavior already exist; the work is to expose that behavior safely at the typed Rust SDK layer.

Python, Node.js, Go, and worker-plugin APIs do not need to change. Existing synchronous native-plugin registrations should remain supported.

Alternatives considered

• Use the raw ABI v3 directly: preserves behavior but requires each plugin to maintain unsafe host interop.
• Increase Tokio worker count: mitigates starvation but does not restore cancellation and scales poorly with concurrent slow calls or idle streams.
• Use a worker plugin: avoids in-process blocking but changes deployment, transport, and packaging.
• Block on a plugin-owned executor from the typed callback: simple, but still occupies Relay workers and disconnects host cancellation from provider transport.

Acceptance criteria

• Safe typed native-plugin APIs support completion-based buffered and streaming middleware.
• Slow plugin I/O does not occupy Relay Tokio workers while awaiting completion.
• Dropping or cancelling a managed call is observable by the plugin and can cancel upstream work.
• Streaming uses bounded backpressure and terminal completion/error semantics.
• Existing synchronous typed callbacks remain compatible.
• Integration tests saturate routed calls at the Relay worker count and verify unrelated managed work remains responsive.
• Integration tests verify buffered and streaming cancellation reaches the plugin task.
• Native plugin documentation explains when to use synchronous versus completion-based callbacks.