fix(llm): abandon a model stream that stops producing events

[devin-ai-integration]

Summary

Stacked on #977 → #976 (same wrapper), so review those first.

A turn can stream a few tokens and then go silent forever. Nothing catches that today: LLM_TIMEOUT becomes httpx's read timeout, which resets on every byte received — provider keepalive/progress frames keep the connection "active" while zero content is generated, so the agent parks indefinitely on a dead generation with no error and no retry.

New LLM_STREAM_IDLE_TIMEOUT (default 300s, 0 disables) bounds the gap between events, not the request:

event = await asyncio.wait_for(iterator.__anext__(), timeout)
# on timeout: close the stream, raise TimeoutError("model stream produced no event for Ns")

Applied in _TurnGuardModel.stream_response, i.e. around the raw provider stream — deliberately not around the run loop's event stream, where a legitimately long-running tool call also produces no events and would be killed. TimeoutError is already classified transient by _is_transient_model_error, so the existing bounded retry replays the turn instead of failing the agent.

Repro

tests/test_stream_idle_timeout.py points the real chat-completions model at a gateway that streams "Now"/" spawning" and then stops writing:

# today: no bound on the gap — the drain is still waiting when the test gives up
with pytest.raises(TimeoutError):
    await asyncio.wait_for(_drain(gateway, idle_timeout=0), timeout=2)

# with the watchdog: the turn is abandoned in ~1s instead of hanging
with pytest.raises(TimeoutError, match="produced no event"):
    await _drain(gateway, idle_timeout=1)

Plus a test that a steadily-producing stream passes through untouched, and one for the setting.

Link to Devin session: https://app.devin.ai/sessions/dad023e379e942f287bcf6822463b7a4
Requested by: @0xallam

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[greptile-apps]

Greptile Summary

The PR adds a configurable watchdog that abandons provider streams when they stop producing events, allowing the existing retry path to replay stalled turns. The latest changes disable that watchdog around non-streaming model wrappers, resolving the previously reported false timeout on valid long-running non-streaming requests.

• Adds LLM_STREAM_IDLE_TIMEOUT, defaulting to 300 seconds with zero disabling it.
• Applies event-gap timeouts around raw provider streams.
• Preserves configured idle timeouts for genuinely streaming models while disabling them for _NonStreamingModel.
• Adds stalled-stream, live-stream, configuration, and non-streaming regression coverage.

Confidence Score: 5/5

The PR appears safe to merge.

The previously reported non-streaming timeout is fixed because the provider now sets the idle timeout to zero whenever it wraps an ordinary model in _NonStreamingModel, while subscription and other streaming routes retain the configured watchdog; no blocking failure remains.

Important Files Changed

Filename	Overview
strix/config/models.py	Adds stream-idle enforcement and correctly resets the watchdog to zero when _NonStreamingModel is selected, fixing the prior non-streaming timeout issue.
strix/config/settings.py	Adds a non-negative environment-configurable stream idle timeout with a 300-second default.
tests/test_stream_idle_timeout.py	Covers stalled and active streams, configuration propagation, and watchdog disablement for non-streaming models.
pyproject.toml	Adds narrowly scoped lint exceptions for the new integration-style test module.

_{Reviews (2): Last reviewed commit: "fix(llm): abandon a model stream that st..." | Re-trigger Greptile}

[devin-ai-integration]

@greptile