core: release the execute-ordering ticket refused by the shutdown gate

[Yaraslaut]

Fixes #348.

The defect

handleImpl takes a per-model execute-ordering ticket on the transport thread, before posting to the pool (remote.hpp:354). dispatchMessage's shutdown gate then returned before dispatchExecute — the only place a ticket was released — so an execute refused with err "server shutting down" dropped its ticket without releasing it.

Both remote.hpp and docs/spec/core/backend.md documented this as a benign exception to the "every path that took a ticket must release it" rule, on the grounds that beginShutdown() is irreversible, so every later execute is refused at the same gate and none reaches awaitExecuteTurn.

That reasoning is wrong, and it misses the window the ordering gate exists because of. Tickets are taken in send order on the transport thread, but the pool may run the two posted tasks in either order — so a later ticket can pass the gate while the earlier one is still upstream of it, and be parked in awaitExecuteTurn (a cv.wait with no deadline) by the time the earlier one is refused.

What it actually costs

Three things, not a leaked map entry:

• the later caller never receives a reply at all — or a spurious err "timeout" for a call that never ran, where LimitPolicy::executeTimeout is configured;
• a pool worker is blocked for the rest of the process's life;
• drainedWithin() can never succeed, because _inFlightExecutes is incremented immediately before that wait — so the defect breaks the very graceful-shutdown sequence during which it fires.

Measured, on this branch's parent:

A (ticket 0) replied: yes -> {"kind":"err",...,"message":"server shutting down"}
B (ticket 1) replied: NO
drainedWithin(3s) = FALSE (took 3000 ms)

The change

dispatchMessage's shutdown branch releases the ticket before replying. The two comments and the spec section that presented the old "benign" analysis are corrected to match — the release rule now holds without exception, which is what makes the rest of that spec section true.

The test

tests/test_remote_execute_ordering.cpp gains a shutdown-gate case that forces the interleaving rather than racing for it: a wrapping executor (HoldOnePostExecutor) holds back one request's dispatch task, so "the later ticket passed the gate, the earlier one did not" is decided, not hoped for. health().inFlight == 1 is the deterministic signal that the later request is parked in the wait, rather than a sleep hoping that it is.

This matters — the natural window is nanoseconds wide, and 400 jittered attempts did not hit it. A probabilistic test here would have been a control that passes while measuring nothing.

Because a regression strands a pool worker in a deadline-less wait, ~ThreadPoolExecutor would block forever in join() and turn a clean assertion failure into a whole-binary hang. The test deliberately leaks its fixture on the failing path instead.

Fails before, passes after — verified in both directions:

# before
tests/test_remote_execute_ordering.cpp:463: FAILED:
  REQUIRE( bCompleted )
with expansion:
  false

# after
All tests passed (33 assertions in 3 test cases)

Verification

• Full suite green under Clang: 1356/1356.
• Rebuilt and re-run under GCC (build/gcc-debug): green.
• The original standalone reproduction from triage — a different mechanism (a 6 MB body as the timing lever rather than executor interception) — also now passes, so the fix is confirmed against two independent reproductions.
• check_spec_citations.sh, check_test_type_names.sh, check_deprecated_markers.sh: all pass.

Filed separately, not folded in

While fixing this I found and reproduced a second path with the same shape and the same consequence: a throw out of dispatchExecute (a throwing IAuthorizer hook, or missingRequiredFields under RequireDeclaredFields) unwinds past every rejectAndRelease into dispatchMessage's outer catch, which replies but never releases. This PR does not fix that — per AGENTS.md it is filed as #351 rather than folded in.

Two different paths have now missed the same per-call-site convention, which is the argument for the structural fix (an RAII ticket holder) that #351 records. Deliberately out of scope here.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!