Skip to content

fix(provider): add NVIDIA NIM DeepSeek request compatibility#37833

Open
fuselayer wants to merge 2 commits into
anomalyco:devfrom
fuselayer:fix/nvidia-nim-request-shape
Open

fix(provider): add NVIDIA NIM DeepSeek request compatibility#37833
fuselayer wants to merge 2 commits into
anomalyco:devfrom
fuselayer:fix/nvidia-nim-request-shape

Conversation

@fuselayer

Copy link
Copy Markdown

Issue for this PR

Closes #24264

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

DeepSeek V4 models on NVIDIA NIM (deepseek-ai/deepseek-v4-flash, deepseek-ai/deepseek-v4-pro) hang instead of streaming a response. NIM only streams reasoning tokens for these models when the request body includes chat_template_kwargs. Since the AI SDK drops unknown top-level config keys, there's no way to set this from opencode.json, so the field never reaches the wire and the request stalls.

Two changes in transform.ts:

  1. In options(), for the nvidia provider on @ai-sdk/openai-compatible, I set chat_template_kwargs: { thinking: true, reasoning_effort: "high" } for deepseek-v4 models. This lands directly on the request body (same pattern as the existing alibaba-cn enable_thinking block right above it), so it isn't stripped by the SDK.

  2. In normalizeMessages(), I remap any developer role message to system for the nvidia provider. NIM returns a 500 on the developer role once chat_template_kwargs is present, so without this the first change just turns the hang into a 500.

I confirmed the field names against NVIDIA's own DeepSeek V4 docs (they use chat_template_kwargs with thinking, not enable_thinking, which is a different NIM model family).

How did you verify your code works?

Ran the patched build locally (bun run dev:desktop) against NVIDIA NIM with my own API key.

  • Before: deepseek-v4-flash hung until timeout.
  • After: it streams normally.
  • No more developer-role 500s.
  • The only errors I hit afterward were NVIDIA-side quota (ResourceExhausted), which confirms the request is actually reaching the backend now instead of stalling on malformed body.

I couldn't re-verify against NVIDIA-hosted Kimi because that model returns a 404 "function not found for account" for my key right now, so I scoped this PR to DeepSeek only.

Screenshots / recordings

N/A — not a UI change.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

@fuselayer
fuselayer force-pushed the fix/nvidia-nim-request-shape branch from 3160fce to 4399657 Compare July 20, 2026 06:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Nvidia NIM API hangs for DeepSeek v4 reasoning models without chat_template_kwargs

1 participant