Skip to content

0.23.0

Choose a tag to compare

@github-actions github-actions released this 03 Sep 02:08
· 15 commits to main since this release

Every vendor's thinking dialect, in one table — and a preflight that fires the real shot

Two halves of the same bug.

The bug

thinking_wire() was the one place in the repo that branched on the model name, and it knew three vendors. Everything else fell through to DeepSeek's private wire format:

out["reasoning_effort"] = lv
out["extra_body"] = {"thinking": {"type": "enabled"}}

The OpenAI SDK flattens extra_body keys to the top level of the request body. So any compatible endpoint without a thinking field 400s on the spot — Unknown name "thinking": Cannot find field. Patching one vendor at a time was never going to end, because the default branch itself was wrong: one vendor's private spelling was being treated as everyone's default.

And the preflight couldn't see it. The old probe sent no thinking dialect, no tools, and didn't stream. An endpoint that rejects your reasoning dialect — or doesn't support tool calling at all — got a green light in the status bar. You found out when the conversation turned red.

pen/providers.py — one table, nine vendors

The single definition point for vendor → four thinking levels. Every cell is a literal wire payload, so nothing is computed at request time.

Vendor off low / medium / high
Generic OpenAI-compatible (nothing) bare reasoning_effortnever extra_body
DeepSeek thinking: disabled effort + thinking: enabled, high → max
GLM 5.3 forced; others disabled low / high / max
Celeris effort: none low / medium / xhigh
Google 3+ and 2.5 Prolow; 2.5 Flash → none effort passed through
OpenAI reasoning models → low; non-reasoning → nothing at all passed through
Kimi K3 → low; K2.7 → enabled; K2.x → disabled K3 low / high / max
Meta effort: minimal (none 400s) low / medium / xhigh
OpenRouter (nothing) bare effort

Model names are matched with care that was earned the hard way:

  • llama does not mean Meta. A llama endpoint on Together, Groq, Ollama, or vLLM has no dialect relationship with Muse Spark.
  • Gemini 2.5 Pro cannot turn thinking off (Google: "N/A: Cannot disable thinking", minimum budget 128). Only Flash and Flash-Lite accept thinkingBudget=0.
  • gpt-5-chat-latest and gpt-5-pro are not reasoning models — sending them reasoning_effort is a 400.
  • openai/o3-mini — gateways prefix the model name, and a pattern anchored at the start of the string misses it silently.

Two deliberate behavior changes: DeepSeek's off now sends an explicit disabled (per DeepSeek's docs, sending nothing means thinking enabled at high — so the old "off" was full-throttle thinking), and unrecognized models no longer receive extra_body.thinking at all.

The preflight now fires the real shot

It calls llm_create_kwargs() itself — the same function the main conversation uses — so the probe carries tools, streaming, stream_options, and your thinking dialect. When the main request grows a new field, the probe grows it too.

max_tokens=1 is gone. That was a parameter we invented to save money, and reasoning models often reject a minimum output that small — so the preflight was manufacturing 400s you'd never hit and then reporting them as your configuration problem. Cost is now controlled by hanging up after the first frame. A green light is exactly one upstream call.

When the full shape fails with a 400, it strips one thing at a time to find out which thing:

① full shape          ok → green
   not a 400 (401/404/unreachable) → report it, stripping won't help
② − thinking dialect  ok → "this endpoint doesn't accept our reasoning-effort dialect"
③ − tools             ok → "this endpoint doesn't support tool calling"
④ − stream_options    ok → "this endpoint doesn't report token usage"
⑤ − stream            ok → "this endpoint doesn't support streaming"

The quoted error is always from shot ①. Later shots are situations we created to locate the problem — if one of them breaks in a different way, the ladder stops rather than blaming you for a failure you never hit.

New in settings: a Provider dropdown

Common settings now lead with Provider, above Base URL. Each option carries a hint line: the official endpoint, what its model names look like, and whether thinking can be switched off. Fast Mode has its own.

  • Defaults to Automatic (matched from the model name), so upgrading changes nothing — the request body doesn't even gain a key.
  • Picking a vendor pre-fills Base URL, but never overwrites an address you typed. It only fills in when the field is empty or still holds another vendor's default. People routing through a gateway need this dropdown the most, and their address belongs to no vendor.
  • Changing the provider re-runs the preflight immediately.

If the status bar says your endpoint rejects the reasoning dialect and you don't know which vendor it really is, pick Generic OpenAI-compatible — it's the only option guaranteed to send no extra_body at any level.

Full Changelog: 0.22.5...0.23.0