Skip to content

v1.42.0

Choose a tag to compare

@myzie myzie released this 23 Oct 17:58
· 13 commits to main since this release
50455f2

Highlights

Claude can now see a token budget for a whole turn. max_tokens caps each
response, but the model cannot see it, so a long thinking pass can use the
whole cap and write no answer. Set ModelSettings.TaskBudget and the model
sees the budget count down as it thinks, calls tools, reads tool results, and
writes, and it paces itself to finish within the budget. The budget is
advisory: a turn can run past it, and MaxTokens still caps each response.
Dive sends the Anthropic beta header for you.

ModelSettings: &dive.ModelSettings{
    ReasoningEffort: llm.ReasoningEffortMedium,
    TaskBudget:      &llm.TaskBudget{Total: 40000},
}

Every model call of one CreateResponse shares the budget. If a PreIteration
hook compacts the turn's history, Dive sends the remaining budget so the
countdown continues. Tool results count against the budget, so size it from
measured turns. Task budgets work on Opus 4.7 and later, Sonnet 5.5, Haiku
5.5, and Fable 5 and 5.1; Anthropic also lists Mythos 5 and 5.1, which Dive
sends the budget to but has not tested. Claude models known to lack support
drop the setting and log a warning, and Claude model IDs that Dive does not
know get it as asked. On Claude, a total below 20,000 is an error. OpenAI and
Gemini drop the setting and log a warning.

The change is additive: existing code needs no changes.

Pull requests

  • feat: Claude task budgets (#345)

Changelog

Added

  • Claude task budgets. ModelSettings.TaskBudget gives the model a token
    budget it can see for the whole turn; unsupported models drop it with a
    warning. See the LLM guide.