Stream not working with custom OpenAI-API-Compatible provider for DeepSeek

[sharpdima]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
• Please do not modify this template :) and fill in all the required fields.

Content

Hi Dify team,

I've set up DeepSeek's official API (https://api.deepseek.com/v1) using the OpenAI-API-Compatible provider in Dify. I've enabled the Stream toggle in the provider settings, but streaming is not being applied—responses still arrive all at once instead of token-by-token.

Additionally, when I try to add a second OpenAI-API-Compatible provider (for a different DeepSeek endpoint), I get the following error after saving:

text
PluginInvokeError: {"args":{},"error_type":"UnboundLocalError","message":"cannot access local variable 'response' where it is not associated with a value"}
My goal is simple: I want streaming to work with my DeepSeek API key directly from DeepSeek's own servers. Could you please advise on how to fix this, or let me know if there's a known issue with streaming on this provider?

Thanks in advance!

[dosubot[bot]]

@sharpdima, there are a few things to check here based on similar reports:

For streaming not working:

1. Reverse proxy buffering — If you're running Dify behind NGINX (or another reverse proxy), make sure buffering is disabled. Add proxy_buffering off; to your NGINX config [1]. Proxy buffering causes the entire response to be collected before forwarding, which defeats streaming.

2. SSE format sensitivity — Dify's streaming parser is strict about Server-Sent Events formatting. Some OpenAI-compatible endpoints don't produce a properly formatted data: prefix (with a space), which silently breaks streaming in the UI even though the backend may receive data correctly [2].

3. Docker networking — If you're running Dify in Docker and pointing to a local or self-hosted endpoint, use host.docker.internal or the actual LAN IP rather than localhost [3].

4. Timeout settings — For DeepSeek specifically, consider increasing timeouts in your .env:

   API_TOOL_DEFAULT_READ_TIMEOUT=300
   HTTP_REQUEST_MAX_READ_TIMEOUT=600
   

   [4]

For the UnboundLocalError on the second provider:

This error (cannot access local variable 'response' where it is not associated with a value) typically means an exception occurred during the HTTP request before the response variable was assigned — likely a connection failure or timeout when validating credentials for the second provider. This often ties back to the same Docker networking or timeout issues above.

Also, if you're on Dify 1.13.1, there was a plugin invocation bug (fixed in 1.13.2) that caused serialization failures with the OpenAI-API-Compatible provider [5]. Upgrading to 1.13.2+ may resolve both issues.

Quick checklist:

• What version of Dify are you running?
• Are you using a reverse proxy (NGINX, Caddy, etc.)?
• Is Dify deployed via Docker?

These details would help narrow down the root cause further.

To reply, just mention @dosu.

---

Share context across your team and agents. Try Dosu.

[richardchen874-sys]

This is a very practical issue, and it is good that you are testing DeepSeek through the OpenAI-compatible provider directly instead of only checking a normal non-streaming response.

For the streaming part, I’d first isolate where the stream is being buffered or lost:

1. Test DeepSeek streaming outside Dify with curl or a small script using stream: true
2. If curl streams correctly, check whether Dify receives chunks but the UI renders them all at once
3. If Dify is behind NGINX / Caddy / Cloudflare, disable response buffering for SSE
4. Confirm the upstream response uses proper SSE format, especially data: ... chunks
5. Check whether the OpenAI-compatible provider plugin is actually sending stream: true in the final request body

For the second provider error, response being unbound usually means validation failed before an HTTP response was assigned, so I’d check endpoint reachability, timeout, Docker network access, and Dify/plugin version first.

For Dify + DeepSeek, the first successful chat completion is not enough. Streaming, provider validation, timeout behavior, and fallback endpoint setup all need to be tested separately.

If you can share whether curl streaming works directly against https://api.deepseek.com/v1/chat/completions, it would narrow this down a lot.