Skip to content

v0.30.0

Choose a tag to compare

@github-actions github-actions released this 05 Oct 21:47
915c834

New

  • A task can say how much the model thinks. Send "reasoning": {"effort": "low"} with a task, or set Thinking on a harness as the default for its tasks; a task's own level wins. The levels are none, minimal, low, medium, high and xhigh. It works on every base that runs a model with levels (19 of the 20; System One's models have none).
  • Each model shows the levels it really has. Models differ: some cannot be turned off, some are only on or off, some lack the lowest or the highest level. GET /v1/bases lists the levels per model, the harness form offers exactly those, and a model asked for a level it lacks gets the nearest one it has.
  • The task says what happened. Its response carries "reasoning": {"effort": "minimal", "applied": "low"} and the provider's own count of thinking tokens under usage.output_tokens_details.reasoning_tokens.

A task and a harness that set no level behave exactly as before.

Fixes

  • CheetahClaws tasks recorded no usage. Every plain CheetahClaws task on 0.29.4 came back with no token counts (8 of 8). The counts are now taken as the provider sends them.
  • Hermes stopped a task whose first answer was long. A first answer that took more than 90 seconds to arrive was treated as a hung call and stopped. A task asked to write a 9,000 word file as its first step ended incomplete after 93 seconds on 0.29.4 and now completes.
  • A task sent with "backend": "claude-code" was refused. The API accepts a base's id as listed by GET /v1/bases as well as its short backend name.

Checked

Which setting moves which model was measured, not assumed: one question at every level, straight at each provider, read from the provider's own thinking-token count. About 900 calls across Vercel AI Gateway, TokenRouter, OpenRouter, OpenAI, Azure OpenAI, Google AI Studio and Anthropic's Messages API, 66 model ids. A level is sent only where it was seen to be accepted, and a level a provider refuses never fails the task: the call is made again without it and the task says so.

Every base, the same task with no level and at none, low and high. Thinking tokens as the provider counted them, gpt-5.4:

Base No level None Low High
Codex 270 0 248 516
Hermes 63 0 40 516
OpenCode 393 0 313 921
Kilo 478 0 215 513
OpenHands 677 0 287 650
Pi 0 0 230 485
Oh My Pi 0 0 262 640
Qwen Code 0 0 276 1,019
Cline 0 0 298 778
Kimi Code 0 0 242 666
MiniMax Code 0 0 192 512
Grok Build 0 0 44 100
Aider 0 0 313 512
Agent Zero 0 0 314 638
CheetahClaws 0 0 402 797

Gemini CLI (gemini-3.5-flash): 3,158 with no level, 2,531 at low, 4,125 at high. goose and DeepSeek Harness, judged on output tokens: 12, 12, about 310, about 675. Claude Code (claude-haiku-4.5), judged on output tokens because Anthropic gives no thinking count: off is unmistakable (about 750 against 2,000 and more); low below high holds on sums of three runs and not on every pair, since a level reaches that model as a ceiling.

The "No level" column is a finding of its own: on the same model, some bases ask for thinking by themselves and some do not. A level set on the harness or the task replaces whatever the base would have asked.

On this release's built candidate, six bases reran the table and passed. On a candidate image carrying the same code: the Thinking control was checked on the harness settings page at four screen widths, saved, reopened, and a task on that harness recorded the level it was given; 25 of 25 ordinary conversation checks on five bases with no level asked; all 15 plugin checks on three bases.

Not measured: Anthropic's API called directly (its rules were read through an aggregator's pass-through), Bedrock, Vertex, and custom endpoints. On a connection that was not measured a level is not sent, and the task says "applied": "default".

docker run -d -p 3000:3000 -v hr-data:/data harnessrouter/harnessrouter:0.30.0

Full Changelog: v0.29.4...v0.30.0