feat(llm): opt-in LLM_DISABLE_STREAMING for non-streaming OpenAI-compatible endpoints

[devin-ai-integration]

Summary

Some OpenAI-compatible gateways don't support streaming (Server-Sent Events), or deliver it unreliably — dropping structured tool_calls deltas, or stalling mid-stream so the whole turn waits out the read timeout. The SDK run loop Strix uses (Runner.run_streamed) only ever issues streamed requests, so against such a gateway every turn fails, with no way to opt out.

This adds an opt-in LLM_DISABLE_STREAMING=true that makes Strix talk to the model without streaming, while keeping the streamed run loop (and therefore tool execution, usage accounting, steering, and the TUI event flow) unchanged.

How: StrixProvider.get_model wraps the resolved model in a small _NonStreamingModel adapter. Its stream_response issues one non-streaming get_response (stream:false on the wire) and replays the completed result as a single terminal response.completed event:

class _NonStreamingModel(Model):
    async def stream_response(self, *args, **kwargs):
        response = await self._inner.get_response(*args, **kwargs)  # stream:false on the wire
        yield _completed_stream_event(response, self._inner.model)  # one response.completed event

This is safe because the run loop builds its authoritative per-turn ModelResponse solely from the response.completed event and then runs the normal get_single_step_result_from_response(...) path — which executes tools and emits run items from final_response.output exactly as it would for a real stream. So a single completed event carrying the full output + usage is sufficient; the only difference is that per-token deltas no longer stream to the UI (inherent when the gateway can't stream).

This mirrors the approach the wider ecosystem converged on for the same problem (e.g. the AI SDK's simulateStreamingMiddleware: call the non-streaming generate, replay as a simulated stream) — an explicit opt-in rather than auto-detection.

Notes / scope:

• Default behavior is unchanged (disable_streaming=False); streaming remains the default.
• Subscription (ChatGPT) models are always streamed and have no non-streaming mode, so they are intentionally not wrapped.
• get_response (the non-streamed path, e.g. dedupe) is a plain pass-through.

Changes

• strix/config/settings.py: new LlmSettings.disable_streaming field (LLM_DISABLE_STREAMING).
• strix/config/models.py: _NonStreamingModel adapter + _completed_stream_event/_response_usage helpers; wired into StrixProvider.get_model.
• tests/test_disable_streaming.py: a local gateway that rejects stream:true but answers non-streamed requests (including a structured tool call), proving the stock model fails against it while the wrapper succeeds — surfacing the tool call, propagating usage, and never sending stream:true. Plus a full Runner.run_streamed integration test (tool call → tool result → final answer over two non-streaming turns) and unit tests for the get_model wrap/no-wrap and subscription-not-wrapped cases.

Testing

• pytest tests/test_disable_streaming.py — 8 passed.
• ruff, ruff-format, mypy, bandit clean via pre-commit on the changed files. (Preexisting mypy findings in strix/interface/tui/app.py are unrelated and present on main.)

Link to Devin session: https://app.devin.ai/sessions/54414ea3ecb34ade9189534e5c690969
Requested by: @0xallam

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[greptile-apps]

Greptile Summary

Adds an opt-in non-streaming adapter for OpenAI-compatible endpoints while preserving the SDK’s streamed agent loop.

• Adds LLM_DISABLE_STREAMING configuration and conditionally wraps non-subscription models.
• Converts completed non-streaming responses into terminal stream events with output and usage data.
• Adds direct adapter, provider-selection, and multi-turn run-loop coverage.
• Updates the Ruff override for the new HTTP-handler tests.

Confidence Score: 5/5

The PR appears safe to merge.

The previous run-loop testing gap is resolved by exercising the real streamed runner, executing the synthetic tool call, continuing to a second model turn, and asserting the final output; no blocking failure remains.

Important Files Changed

Filename	Overview
strix/config/models.py	Adds the non-streaming model adapter, completed-event conversion, usage propagation, and conditional provider wrapping.
strix/config/settings.py	Adds the opt-in LLM_DISABLE_STREAMING setting with streaming retained as the default.
tests/test_disable_streaming.py	Covers non-streaming requests, tool-call and usage propagation, provider wrapping, and the complete multi-turn streamed runner flow.
pyproject.toml	Allows standard-library HTTP handler method names in the new test module.

_{Reviews (2): Last reviewed commit: "test(llm): cover the full run loop again..." | Re-trigger Greptile}

[devin-ai-integration]

@greptile