io_uring: probeSendZC can never detect copy-fallback — SEND_ZC enabled on every host regardless of NIC

[FumingPower3925]

Summary

probeSendZC can never detect the copy-fallback case, so SEND_ZC is enabled on
every host regardless of whether the NIC can actually do zero-copy DMA. The
probe reports SendZCTrueZeroCopy from a loopback socket — a case the
function's own doc comment says must always be SendZCCopyFallback.

Found while auditing whether celeris's io_uring engine would work on riscv64.
It turned out not to be arch-specific at all: it affects amd64 and arm64 today.

The defect

Two problems, one of which makes the other invisible.

1. notifUsageZCCopied has the wrong value.

// engine/iouring/probe.go:73
notifUsageZCCopied = 2 // IORING_NOTIF_USAGE_ZC_COPIED: data was copied, not zero-copied

The kernel UAPI is:

#define IORING_NOTIF_USAGE_ZC_COPIED (1U << 31)

So the test at probe.go:214 inspects bit 1 instead of bit 31:

if notifRes&notifUsageZCCopied != 0 {
    return SendZCCopyFallback, "REPORT_USAGE notification reports IORING_NOTIF_USAGE_ZC_COPIED (kernel did the copy)"
}

It cannot fire, so probeSendZC always falls through to return SendZCTrueZeroCopy.

2. The probe runs on loopback, where the answer is knowable a priori.

// engine/iouring/probe.go:119
ln, err := net.Listen("tcp", "127.0.0.1:0")

And the function's own comment at probe.go:114 states:

On loopback, the result is always SendZCCopyFallback (kernel copies data, no DMA).

So even with the constant corrected, this probe can only ever report
SendZCCopyFallback — it never observes the real NIC. The probe as designed
cannot answer the question it is named for.

Evidence

From a live bench server on the arm64 node (msr1, kernel 7.0.0-30):

INFO SEND_ZC probe result result="true zero-copy"
INFO io_uring engine selected tier=high multishot_accept=true multishot_recv=true \
     provided_buffers=true fixed_files=false send_zc=true

result="true zero-copy" from a loopback probe, which the code documents as
impossible.

Why the fix needs care

completionEntry.Res is int32 (engine/iouring/cqe.go:8), so the untyped
constant 1 << 31 overflows it. A correct comparison needs something like:

const notifUsageZCCopied uint32 = 1 << 31
...
if uint32(notifRes)&notifUsageZCCopied != 0 { ... }

Blast radius

Narrow but real. SEND_ZC is gated at worker.go:3585:

return sendZC && !linked && n >= sendZCMinBytes

— unlinked sends only, at or above sendZCMinBytes. In the benchmark grid that
is essentially get-json-64k and ws-large-echo. Every other cell is unaffected.

worker.go:2201 also disables SEND_ZC at runtime on -EINVAL, so a kernel that
rejects the opcode outright is still handled. The gap is specifically the
"accepted but silently copying" case, which is what REPORT_USAGE exists to detect.

Suggested direction

1. Fix the constant and the signedness (above). Low risk, restores the intent.
2. Decide what the probe should actually measure. Options:
   • Probe over the real bench interface rather than loopback, so the result
     reflects the NIC that will carry traffic; or
   • Keep the loopback probe as a pure functional check (does the NOTIF CQE
     arrive at all — the ENA/AWS case at probe.go:116) and stop inferring
     zero-copy capability from it, deriving that from the device's
     NETIF_F_SG instead.

Worth noting the likely outcome: once the constant is fixed, a loopback probe
will correctly report SendZCCopyFallback and disable SEND_ZC on every host.
That will move get-json-64k and ws-large-echo numbers. That is a correction,
not a regression — but it should land at a version boundary so published results
stay comparable.

Affected

All architectures, current main. Not arch-specific despite how it was found.