`RuntimeError: There is no Stream(gpu, N) in current thread` when calling `/v1/messages` or `/v1/chat/completions` on Qwen3.5 MLX models (MLLM path runs in worker thread)

[luopeixiang]

RuntimeError: There is no Stream(gpu, N) in current thread when calling /v1/messages or /v1/chat/completions on Qwen3.5 MLX models (MLLM path runs in worker thread)

Summary

On Apple Silicon, any model whose name matches the MLLM pattern Qwen3.5- (e.g. mlx-community/Qwen3.5-4B-4bit, mlx-community/Qwen3.5-9B-4bit) fails at inference time for chat-style endpoints with:

RuntimeError: There is no Stream(gpu, 0) in current thread.

(In --continuous-batching mode the error becomes Stream(gpu, 2) at a different site, same root cause.)

/v1/completions (raw prompt) works fine. /v1/chat/completions and /v1/messages both fail. In non-streaming mode /v1/messages currently masks the failure and returns HTTP 200 with empty content and input_tokens/output_tokens = 0, which is a secondary bug.

Environment

• OS: macOS, Apple Silicon (unified memory ~38.7 GB)
• Python: 3.13.12 (cpython-3.13.12-macos-aarch64-none, from uv)
• Installed via: uv tool install git+https://github.com/waybarrios/vllm-mlx.git (latest)
• Models reproduced on:
  • mlx-community/Qwen3.5-4B-4bit
  • mlx-community/Qwen3.5-9B-4bit
• Both are cached locally; running with --offline.

Reproduction

A. Continuous-batching mode → Stream(gpu, 2)

Start server:

vllm-mlx serve mlx-community/Qwen3.5-9B-4bit \
  --port 8000 \
  --continuous-batching \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --mllm-prefill-step-size 8192 \
  --offline

Call:

curl -s http://localhost:8000/v1/messages \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mlx-community/Qwen3.5-9B-4bit",
    "max_tokens": 256,
    "messages": [{"role": "user", "content": "Hello, please introduce yourself."}]
  }'

Client sees HTTP 200 with empty content:

{"id":"msg_...","type":"message","role":"assistant",
 "model":"mlx-community/Qwen3.5-9B-4bit",
 "content":[{"type":"text","text":""}],
 "stop_reason":"end_turn",
 "usage":{"input_tokens":0,"output_tokens":0}}

Server log shows the actual error being swallowed:

INFO  vllm_mlx.mllm_batch_generator: [think_suffix] Detected think tag '<think>' = 2 token(s)
ERROR vllm_mlx.mllm_batch_generator: Failed to process batch of 1 prompts:
      RuntimeError: There is no Stream(gpu, 2) in current thread.
Traceback (most recent call last):
  File ".../vllm_mlx/mllm_batch_generator.py", line 1559, in _next
    new_batch = self._process_prompts(requests)
  File ".../vllm_mlx/mllm_batch_generator.py", line 1310, in _process_prompts
    mx.eval(sampled, logprobs)
RuntimeError: There is no Stream(gpu, 2) in current thread.
WARNING vllm_mlx.mllm_scheduler: Request ... failed during preprocessing
INFO  vllm_mlx.mllm_scheduler: Aborting orphaned MLLM request ...
INFO  "POST /v1/messages HTTP/1.1" 200 OK

B. Simple (non-batched) mode → Stream(gpu, 0) + HTTP 500

Start server with the minimum set of flags:

vllm-mlx serve mlx-community/Qwen3.5-9B-4bit --port 8000 --offline

Same curl call → HTTP 500 Internal Server Error. Server log:

INFO  vllm_mlx.server: [REQUEST] POST /v1/messages ...
INFO  vllm_mlx.models.mllm: MLLM.chat() called with 1 messages
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  ...
  File ".../vllm_mlx/server.py", line 3532, in create_anthropic_message
    output = await _wait_with_disconnect(...)
  File ".../vllm_mlx/server.py", line 2780, in _wait_with_disconnect
    return task.result()
  File ".../vllm_mlx/engine/simple.py", line 520, in chat
    output = await self._run_blocking_serialized(...)
  File ".../vllm_mlx/engine/simple.py", line 245, in _run_blocking_serialized
    return await asyncio.shield(task)
  File ".../asyncio/threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
  File ".../concurrent/futures/thread.py", line 59, in run
    result = self.fn(*self.args, **self.kwargs)
  File ".../vllm_mlx/models/mllm.py", line 1673, in chat
    result = generate(self.model, ...)
  File ".../mlx_vlm/generate.py", line 856, in generate
    for response in stream_generate(model, processor, prompt, ...):
  File ".../mlx_vlm/generate.py", line 727, in stream_generate
    with wired_limit(model, [generation_stream]):
  File ".../contextlib.py", line 162, in __exit__
    self.gen.throw(value)
  File ".../mlx_vlm/generate.py", line 327, in wired_limit
    mx.synchronize(s)
RuntimeError: There is no Stream(gpu, 0) in current thread.

/v1/chat/completions fails in exactly the same way (also routes through mllm.chat()).

C. /v1/completions (raw prompt) works

Same server, different endpoint:

curl -s http://localhost:8000/v1/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"mlx-community/Qwen3.5-9B-4bit","max_tokens":64,"prompt":"hi"}'

Works fine and returns real text. This endpoint does not go through MLLM.chat() → mlx_vlm.generate.stream_generate → wired_limit, which is why it escapes the bug.

[perry2of5]

Broken test on main tests for this: tests/test_batching_deterministic.py::TestBatchGeneratorCleanup::test_batch_generator_closed_after_engine_stop

Looks like maybe the true failure mode is the model itself is not portable across threads after load, so worker-thread stepping is triggering the stream error. I’m implementing a targeted fallback, but it might be more expedient to figure out which commit broke this and go back and fix it there.

[ChristopherA]

Hitting this on text-only Qwen3-Coder-30B-A3B-Instruct-4bit — the bug is wider than the MLLM path.

Environment

• vllm-mlx commit 42d184203ace56068be1266d57dd6c4411e59c62 (current main, the PR fix: streaming tool calls drop for Qwen3.6 bracket format #374 merge commit)
• Python 3.11.15 via pipx (clean install on a machine that had no prior vllm-mlx)
• M4 Max / 128 GB / macOS 26.4.1
• Model: mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit (text-only LLM, not MLLM)

Both server modes hit the same root cause; only the log line differs:

Mode	Log message
BatchedEngine (--continuous-batching --use-paged-cache)	ERROR:vllm_mlx.scheduler:Error in batch generation step: There is no Stream(gpu, 4) in current thread.
SimpleEngine (no batching flags)	ERROR:vllm_mlx.server:Streaming error, ensuring terminal frame: There is no Stream(gpu, 6) in current thread.

Reproducer

vllm-mlx serve mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit \
  --port 8001 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3 \
  --enable-prefix-cache --max-tokens 4096

# stream:true → empty delta then immediate [DONE], no tokens
curl -N http://127.0.0.1:8001/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit",
       "messages":[{"role":"user","content":"Count: one, two, three"}],
       "max_tokens":30,"temperature":0,"stream":true}'

# stream:false → returns content correctly
curl http://127.0.0.1:8001/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit",
       "messages":[{"role":"user","content":"Count: one, two, three"}],
       "max_tokens":30,"temperature":0,"stream":false}'
# returns: "One, two, three, four, five, ..., fifteen,"

vllm-mlx bench-serve reports all 9 runs as FAIL with gen_tps:0, because bench-serve uses streaming for TTFT measurement and the server-side Streaming error, ensuring terminal frame fires for each request.

PR #399 ("keep MLLM generation on model-load thread") covered the MLLM path; the LLM streaming path appears to need the same thread-affinity fix. The non-streaming code path runs generation directly in the request thread and works in both engine modes.

[waybarrios]

Can't reproduce on commit 42d1842 / M4 Max / macOS 26.1

Tried the exact reproducer from the issue in a clean conda env at the same commit and the server streams normally on both engine modes. No There is no Stream(gpu, N) in current thread error anywhere in the logs.

---

Environment

Hardware / OS

Model:    MacBook Pro (Mac16,6, Z1FE000NSLL/A)
Chip:     Apple M4 Max (16 cores: 12P + 4E)
Memory:   128 GB
OS:       macOS 26.1 (25B78) / Darwin 25.1.0 arm64

Code under test

repo:   waybarrios/vllm-mlx
commit: 42d184203ace56068be1266d57dd6c4411e59c62
title:  fix: streaming tool calls drop for Qwen3.6 bracket format (#374)
date:   2026-04-23 21:16:43 -0500

Python env — fresh conda env, no prior vllm-mlx state

conda create -n vllmmlx-issue407 python=3.11.15 -c conda-forge -y
conda activate vllmmlx-issue407
pip install /path/to/vllm-mlx   # install from the 42d1842 checkout

Key MLX versions

mlx           0.31.2
mlx-lm        0.31.3
mlx-metal     0.31.2
mlx-vlm       0.4.4
mlx-embeddings 0.1.0
transformers  5.6.2
tokenizers    0.22.2
torch         2.11.0
numpy         2.4.4
vllm-mlx      0.2.9 (from commit 42d1842)
python        3.11.15

Full pip freeze (click to expand)

aiohappyeyeballs==2.6.1
aiohttp==3.13.5
aiosignal==1.4.0
annotated-doc==0.0.4
annotated-types==0.7.0
anyio==4.13.0
attrs==26.1.0
brotli==1.2.0
certifi==2026.4.22
cffi==2.0.0
charset-normalizer==3.4.7
click==8.3.3
cryptography==46.0.7
datasets==4.8.4
dill==0.4.1
fastapi==0.136.1
filelock==3.29.0
frozenlist==1.8.0
fsspec==2026.2.0
gradio==6.13.0
gradio_client==2.5.0
groovy==0.1.2
h11==0.16.0
hf-gradio==0.4.1
hf-xet==1.4.3
httpcore==1.0.9
httpx==0.28.1
httpx-sse==0.4.3
huggingface_hub==1.11.0
idna==3.13
interegular==0.3.3
Jinja2==3.1.6
jsonschema==4.26.0
jsonschema-specifications==2025.9.1
lm-format-enforcer==0.11.3
markdown-it-py==4.0.0
MarkupSafe==3.0.3
mcp==1.27.0
mdurl==0.1.2
miniaudio==1.70
mlx==0.31.2
mlx-embeddings==0.1.0
mlx-lm==0.31.3
mlx-metal==0.31.2
mlx-vlm==0.4.4
mpmath==1.3.0
multidict==6.7.1
multiprocess==0.70.19
networkx==3.6.1
numpy==2.4.4
opencv-python==4.13.0.92
orjson==3.11.8
pandas==3.0.2
pillow==12.2.0
prometheus_client==0.25.0
propcache==0.4.1
protobuf==7.34.1
psutil==7.2.2
pyarrow==24.0.0
pycparser==3.0
pydantic==2.13.3
pydantic-settings==2.14.0
pydantic_core==2.46.3
pydub==0.25.1
Pygments==2.20.0
PyJWT==2.12.1
python-dateutil==2.9.0.post0
python-dotenv==1.2.2
python-multipart==0.0.26
pytz==2026.1.post1
PyYAML==6.0.3
referencing==0.37.0
regex==2026.4.4
requests==2.33.1
rich==15.0.0
rpds-py==0.30.0
safehttpx==0.1.7
safetensors==0.7.0
semantic-version==2.10.0
sentencepiece==0.2.1
shellingham==1.5.4
six==1.17.0
sse-starlette==3.3.4
starlette==1.0.0
sympy==1.14.0
tabulate==0.10.0
tokenizers==0.22.2
tomlkit==0.14.0
torch==2.11.0
torchvision==0.26.0
tqdm==4.67.3
transformers==5.6.2
typer==0.24.2
typing-inspection==0.4.2
typing_extensions==4.15.0
urllib3==2.6.3
uvicorn==0.46.0
vllm-mlx @ file:///Users/waybarrios/Documents/code/vllm-mlx
xxhash==3.6.0
yarl==1.23.0

---

Tests executed

Model: mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit (16 GB, 4 safetensor shards)

Common flags: --port 8010 --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3 --enable-prefix-cache --max-tokens 4096

Reproducer payload (copy of the issue's curl, stream toggled):

{"model":"mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit",
 "messages":[{"role":"user","content":"Count: one, two, three"}],
 "max_tokens":30,"temperature":0,"stream":<true|false>}

#	Engine mode	Extra flags	stream	Outcome
1	BatchedEngine	--continuous-batching --use-paged-cache	true	20 tokens / 3.9 s / 5.1 tok/s (first token 3.7 s)
2	BatchedEngine	same	false	30 tokens / 0.34 s / 88.4 tok/s
3	BatchedEngine	same	true x 5 concurrent	5/5 succeeded, 3480-3763 B each
4	SimpleEngine	(none)	true	64 SSE lines / 52.1 tok/s
5	SimpleEngine	(none)	false	30 tokens / 0.45 s / 67.2 tok/s

Error grep on both server logs:

$ grep -E 'Error|Stream\(gpu|no Stream|Exception|Traceback' server-batched.log
INFO:vllm_mlx.engine_core:[stream_outputs] ... CancelledError after 20 tokens, 3.9s   # <- normal client disconnect
$ grep -E 'Error|Stream\(gpu|no Stream|Exception|Traceback' server-simple.log
(no matches)

Only "error" is the expected CancelledError when curl closes the SSE stream at max_tokens. No Metal/stream errors.

Sample of a working stream chunk (BatchedEngine, stream:true):

data: {"choices":[{"delta":{"reasoning_content":"One"}}],...}
data: {"choices":[{"delta":{"reasoning_content":","}}],...}
data: {"choices":[{"delta":{"reasoning_content":" two"}}],...}
...

Note on client-side parsing: with --reasoning-parser qwen3, tokens arrive under delta.reasoning_content, not delta.content. A client that only inspects delta.content will see what looks like an "empty delta then [DONE]" stream even though data is flowing normally.

[perry2of5]

I can reproduce it with git remote -v && git fetch upstream main && git rebase upstream/main && rm -rf .venv uv.lock && uv sync && uv pip install -e ".[dev,vision]" && source .venv/bin/activate && uv run pytest tests/test_batching_deterministic.py::TestBatchGeneratorCleanup::test_batch_generator_closed_after_engine_stop

upstream	git@github.com:waybarrios/vllm-mlx.git (fetch)
upstream	git@github.com:waybarrios/vllm-mlx.git (push)
From github.com:waybarrios/vllm-mlx
 * branch            main       -> FETCH_HEAD
Current branch main is up to date.
Using CPython 3.13.11
Creating virtual environment at: .venv
Resolved 284 packages in 92ms
Installed 104 packages in 329ms
 + aiohappyeyeballs==2.6.1
 + aiohttp==3.13.5
 + aiosignal==1.4.0
 + annotated-doc==0.0.4
 + annotated-types==0.7.0
 + anyio==4.13.0
 + attrs==26.1.0
 + audioop-lts==0.2.2
 + brotli==1.2.0
 + certifi==2026.4.22
 + cffi==2.0.0
 + charset-normalizer==3.4.7
 + click==8.3.3
 + cryptography==46.0.7
 + datasets==4.8.4
 + dill==0.4.1
 + fastapi==0.136.1
 + filelock==3.29.0
 + frozenlist==1.8.0
 + fsspec==2026.2.0
 + gradio==6.13.0
 + gradio-client==2.5.0
 + groovy==0.1.2
 + h11==0.16.0
 + hf-gradio==0.4.1
 + hf-xet==1.4.3
 + httpcore==1.0.9
 + httpx==0.28.1
 + httpx-sse==0.4.3
 + huggingface-hub==1.11.0
 + idna==3.13
 + interegular==0.3.3
 + jinja2==3.1.6
 + jsonschema==4.26.0
 + jsonschema-specifications==2025.9.1
 + lm-format-enforcer==0.11.3
 + markdown-it-py==4.0.0
 + markupsafe==3.0.3
 + mcp==1.27.0
 + mdurl==0.1.2
 + miniaudio==1.70
 + mlx==0.31.2
 + mlx-embeddings==0.1.0
 + mlx-lm==0.31.1
 + mlx-metal==0.31.2
 + mlx-vlm==0.4.4
 + mpmath==1.3.0
 + multidict==6.7.1
 + multiprocess==0.70.19
 + networkx==3.6.1
 + numpy==2.2.6
 + opencv-python==4.13.0.92
 + orjson==3.11.8
 + packaging==26.1
 + pandas==2.3.3
 + pillow==12.2.0
 + prometheus-client==0.25.0
 + propcache==0.4.1
 + protobuf==6.33.6
 + psutil==7.2.2
 + pyarrow==24.0.0
 + pycparser==3.0
 + pydantic==2.13.3
 + pydantic-core==2.46.3
 + pydantic-settings==2.14.0
 + pydub==0.25.1
 + pygments==2.20.0
 + pyjwt==2.12.1
 + python-dateutil==2.9.0.post0
 + python-dotenv==1.2.2
 + python-multipart==0.0.26
 + pytz==2026.1.post1
 + pyyaml==6.0.3
 + referencing==0.37.0
 + regex==2026.4.4
 + requests==2.33.1
 + rich==15.0.0
 + rpds-py==0.30.0
 + safehttpx==0.1.7
 + safetensors==0.7.0
 + semantic-version==2.10.0
 + sentencepiece==0.2.1
 + setuptools==80.10.2
 + shellingham==1.5.4
 + six==1.17.0
 + sse-starlette==3.3.4
 + starlette==0.52.1
 + sympy==1.14.0
 + tabulate==0.10.0
 + tokenizers==0.22.2
 + tomlkit==0.14.0
 + torch==2.10.0
 + torchvision==0.25.0
 + tqdm==4.67.3
 + transformers==5.6.2
 + typer==0.24.2
 + typing-extensions==4.15.0
 + typing-inspection==0.4.2
 + tzdata==2026.1
 + urllib3==2.6.3
 + uvicorn==0.46.0
 + vllm-mlx==0.2.9 (from file:///Users/tperry/code/llm/vllm-mlx)
 + xxhash==3.6.0
 + yarl==1.23.0
Resolved 115 packages in 38ms
      Built vllm-mlx @ file:///Users/tperry/code/llm/vllm-mlx
Prepared 1 package in 347ms
Uninstalled 1 package in 0.83ms
Installed 12 packages in 38ms
 + black==26.3.1
 + iniconfig==2.3.0
 + librt==0.9.0
 + mypy==1.20.2
 + mypy-extensions==1.1.0
 + pathspec==1.1.0
 + platformdirs==4.9.6
 + pluggy==1.6.0
 + pytest==9.0.3
 + pytokens==0.4.1
 + ruff==0.15.11
 ~ vllm-mlx==0.2.9 (from file:///Users/tperry/code/llm/vllm-mlx)
================================================================================================================================================= test session starts =================================================================================================================================================
platform darwin -- Python 3.13.11, pytest-9.0.3, pluggy-1.6.0 -- /Users/tperry/code/llm/vllm-mlx/.venv/bin/python3
cachedir: .pytest_cache
rootdir: /Users/tperry/code/llm/vllm-mlx
configfile: pytest.ini (WARNING: ignoring pytest config in pyproject.toml!)
plugins: anyio-4.13.0
collected 1 item                                                                                                                                                                                                                                                                                                      

tests/test_batching_deterministic.py::TestBatchGeneratorCleanup::test_batch_generator_closed_after_engine_stop FAILED                                                                                                                                                                                           [100%]

====================================================================================================================================================== FAILURES =======================================================================================================================================================
_______________________________________________________________________________________________________________________ TestBatchGeneratorCleanup.test_batch_generator_closed_after_engine_stop _______________________________________________________________________________________________________________________
tests/test_batching_deterministic.py:486: in test_batch_generator_closed_after_engine_stop
    assert bg is not None, "BatchGenerator should exist after generation"
E   AssertionError: BatchGenerator should exist after generation
E   assert None is not None
------------------------------------------------------------------------------------------------------------------------------------------------ Captured stderr setup ------------------------------------------------------------------------------------------------------------------------------------------------
Fetching 6 files: 100%|██████████| 6/6 [00:00<00:00, 12116.43it/s]
-------------------------------------------------------------------------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------------------------------------------------------------------------
ERROR    vllm_mlx.scheduler:scheduler.py:2504 Error in batch generation step: There is no Stream(gpu, 2) in current thread.
Traceback (most recent call last):
  File "/Users/tperry/code/llm/vllm-mlx/vllm_mlx/scheduler.py", line 2462, in step
    result = self.batch_generator.next()
  File "/Users/tperry/code/llm/vllm-mlx/.venv/lib/python3.13/site-packages/mlx_lm/generate.py", line 1329, in next
    return self._next()
           ~~~~~~~~~~^^
  File "/Users/tperry/code/llm/vllm-mlx/.venv/lib/python3.13/site-packages/mlx_lm/generate.py", line 1257, in _next
    batch = self._process_prompts(prompts)
  File "/Users/tperry/code/llm/vllm-mlx/vllm_mlx/scheduler.py", line 152, in _patched_process_prompts
    batch = _orig_process_prompts(prompts)
  File "/Users/tperry/code/llm/vllm-mlx/.venv/lib/python3.13/site-packages/mlx_lm/generate.py", line 1107, in _process_prompts
    mx.eval([c.state for c in prompt_cache])
    ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: There is no Stream(gpu, 2) in current thread.

WARNING  vllm_mlx.scheduler:scheduler.py:2406 [generation_error_recovery] aborted 1 running requests, batch generator closed, Metal cache cleared
=============================================================================================================================================== short test summary info ===============================================================================================================================================
FAILED tests/test_batching_deterministic.py::TestBatchGeneratorCleanup::test_batch_generator_closed_after_engine_stop - AssertionError: BatchGenerator should exist after generation
================================================================================================================================================= 1 failed in 14.89s

[waybarrios]

PR #411 addresses part of this. It moves EngineCore._engine_loop's scheduler step onto the event-loop thread (same pattern PR #399 used for MLLM), adds tests/test_engine_core_stream_safety.py as a regression guard, and wires the stream-sensitive tests into the apple-silicon CI.

With the patch applied, Llama-3.2-1B and 3B (reliably broken before) now pass the AsyncEngineCore path. The vllm-mlx serve --continuous-batching path with Llama still reproduces the Stream(gpu, N) error though, so there is a second cross-thread code path in the server startup (model loaded via run_blocking_startup_work on an asyncio executor thread, step runs on the event-loop thread) that #411 does not cover. Working on a follow-up for that.

@ChristopherA if you have a minute, would you give #411 a try against your Qwen3-Coder-30B-A3B setup on macOS 26.4.1 and let me know if the error signature changes?

[waybarrios]

@perry2of5 could you check the following PR #411?

[waybarrios]

@Thump604 @janhilgard Please need an extra hand on this.

[waybarrios]

Ran vllm-mlx bench-serve on a fresh conda env (Python 3.11.15, mlx 0.31.2, mlx-lm 0.31.3, M4 Max 128 GB, macOS 26.1) with both commits from #411 applied. Everything looks good.

model	tok/s
mlx-community/Qwen3-0.6B-8bit	405.1
mlx-community/Llama-3.2-3B-Instruct-4bit	205.6
mlx-community/Qwen3-30B-A3B-4bit	121.5
mlx-community/gemma-4-e2b-it-mxfp4	143.3
mlx-community/Qwen3.5-4B-MLX-4bit	111.3

bench-serve --prompts short --concurrency 1 --max-tokens 50 --repetitions 3 --warmup 1 --extra-body '{"temperature":0}'. No Stream(gpu) errors in any run.

[perry2of5]

with PR 411
tests/test_continuous_batching.py::TestContinuousBatchingIntegration::test_single_request fails
tests/test_lifecycle_server.py::TestLifecycleFailureHandling::test_eager_engine_start_cancellation_cleans_prepared_state[batched-llm] hangs

uv run pytest tests/test_batching_deterministic.py passes -- this failed before