PR-D2 (ADR 0008 Phase D): refactor HTTP shim onto SessionStore

[FluffyAIcode]

Why

Retires the Scheduler + PooledVerifier + SpeculativeEngine machinery from the HTTP shim's request path. Each /v1/chat/completions request is now a single-shot session under SessionStore: CreateSession → AppendTokens(prompt) → Generate → CloseSession. Same semantics as the gRPC RuntimeService surface; ADR 0008 §2.7 deprecation.

Three architectural changes

#	Change	Notes
1	Speculative decoding is no longer applied on the HTTP path	The session-bound runtime is pure AR against the verifier; the proposer is wired into v0.4 alignment work (ADR 0004). Pre-PR-D2 the HTTP shim used SpeculativeEngine; post-PR-D2 it's roughly the same speed as transformers-vanilla AR. Migrate to gRPC for v0.3's full perf story.
2	Admission control is now asyncio.Semaphore	REJECT vs QUEUE policy with queue_max_wait_s is preserved (queue_max_wait_s=0 means wait forever); the in-flight slab-pool bookkeeping moved into SessionStore. The Scheduler module + integration tests stay (used by other callers), just no longer wired to HTTP.
3	ADR 0008 §2.7 deprecation headers	Stamped onto every response by a new _DeprecationHeadersMiddleware: Deprecation: true, Sunset: Wed, 31 Dec 2025 00:00:00 GMT, Link: </docs/adr/0008-...>; rel="successor-version".

Files

Direction	File	Δ
Rewritten	inference_engine/server/app.py	+330 / −300 net; create_app(verifier, config, *, slab_pool=None, model_id_label=None); new _DeprecationHeadersMiddleware; route handler runs SessionStore + AppendTokensCoordinator + GenerationCoordinator with sync gen wrapped in asyncio.to_thread for disconnect-poll responsiveness
Deleted	inference_engine/scheduler/pooled_verifier.py	−175
Updated	inference_engine/scheduler/__init__.py	dropped PooledVerifier from __all__
Rewritten	scripts/serve.py	_build_engine → _build_verifier; banner says "DEPRECATED HTTP shim", points at gRPC entrypoint; --block-size / --num-diffusion-steps flags retained for CLI compat but documented as ignored
Deleted	tests/inference_engine/scheduler/test_pooled_verifier.py	−272; PR-N1's exemption was precisely because PR-D2 retires this module
Added	tests/inference_engine/server/test_grpc_app.py	+132, 3 tests covering grpc_app's success paths (test_append_tokens_session_not_found_returns_not_found, test_append_tokens_success_returns_response, test_generate_yields_history_truncated_then_done) — previously covered by deleted-in-PR-N3 FakeVerifier-backed tests
Updated	tests/integration/test_http_shim_real.py	Fixture: real_speculative_engine → real_speculative_engine._decoder.verifier. Asserts: state.engine.model_id_label → state.model_id_label.
Updated	.github/workflows/ci.yaml	dropped pooled_verifier.py from --include= filter
Added	scripts/review_pr_d2_on_mac.sh	Mac M4 reviewer aid

Linux verification

PYTHONPATH=.:sdks/python coverage run -m pytest <Linux gate paths>:
  476 passed (was 473 on main; +3 net = added 3 grpc_app success-path tests).
  100% coverage on 915 stmts (was 987 on main; -72 net = deleted PooledVerifier).

Mac M4 evidence (REQUIRED for merge)

This is the single most invasive PR in the v0.3 sequence — it rewrites the deprecated HTTP shim's entire request path. The integration suite's test_http_shim_real.py is the binding gate. Reviewer runs:

bash scripts/review_pr_d2_on_mac.sh
git add results/platform-tests/pr-d2-mac-*
git commit -m "Mac M4 review evidence for PR-D2"
git push

Acceptance: all integration tests pass against real Qwen3-0.6B, including the now-rewired test_http_shim_real.py which exercises chat-completions (streaming + non-streaming), auth, /healthz, /metrics, /v1/models against the new SessionStore-driven path.

Stack

PR-D2 is branched off post-N1..N4 main. Independent of PR-E2 (#57) which adds CI workflow YAML; the two can merge in either order.

Next PR

v0.4 brings the proposer back into the session-bound path: PR-V0.4-A wires SparseLogitsProposer into a new SpeculativeAppendTokensCoordinator (or extends the existing one) so speculative decoding is restored on both gRPC and HTTP paths. The ADR 0001 / 0004 alignment training feeds into that work.

Reviewer checklist

• create_app signature change is documented in PR description (was engine, config, pool=None; now verifier, config, *, slab_pool=None, model_id_label=None).
• PooledVerifier is gone from the scheduler package; no callers remain anywhere in the repo.
• Deprecation / Sunset headers are visible in every HTTP response (use curl -I against a running shim to verify).
• Mac M4 evidence committed to this branch — pr-d2-mac-integration-tests-*.json shows all integration tests passing against real Qwen3-0.6B.
• serve.py's --block-size / --num-diffusion-steps flags are retained but documented as ignored (don't silently drop them — existing scripts may pass them).