reasoning-budget: Implementing thinking-budget mechanism to control thought process

[laurencehardman]

Overview

Extends the reasoning-budget sampler with two new stages around the existing hard cutoff: an intro message forced when the reasoning block starts (announcing the token budget via a {budget} placeholder), and a soft warning forced at a configurable fraction of the budget, injected at the next newline boundary.

Both are opt-in via
--reasoning-budget-soft-ratio/-message, and
--reasoning-budget-intro-message, with per-request overrides.

Additional information

The primary driver for this change is to rein-in / restrict the tendency of (particularly low-temperature) models from entering a thought-loop where their thought process either starts becoming repetitive, or spirals into nonsense - as does Qwen3.6-27B at lower-temperature settings. This mechanism forces the LLM to determiniscally ration its thinking budget and structure its thoughts with the budget in mind, as well as cut itself off (with both a soft and hard limit) once that budget has been reached.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: Yes - Claude Sonnet to assist with extending the boilerplate reasoning-budget module, test generation, build, and final sense-check of the implementation.

[coder543]

Have you run any standard evals to see how this affects both scores and token consumption?

Statistically over a large number of example problems/requests, does this actually force the model to “deterministically ration” its token budget and avoid hitting the hard reasoning budget limit? If it is deterministic, it should be measurable.

This PR doesn’t have any evidence that it actually helps the model at all, and as others pointed out on Reddit, even tiny formatting errors in the chat template can significantly impede model performance. A model that isn’t trained to talk to itself about token budgets seems unlikely to benefit from that, but I’d love to see data.

[thekk1]

Have you run any standard evals to see how this affects both scores and token consumption?

Statistically over a large number of example problems/requests, does this actually force the model to “deterministically ration” its token budget and avoid hitting the hard reasoning budget limit? If it is deterministic, it should be measurable.

This PR doesn’t have any evidence that it actually helps the model at all, and as others pointed out on Reddit, even tiny formatting errors in the chat template can significantly impede model performance. A model that isn’t trained to talk to itself about token budgets seems unlikely to benefit from that, but I’d love to see data.

I have the same concerns. The initial message should be deleted from this PR because it could break the model quality in an unforeseeable way.

The soft limit "reasoning-budget-soft-message" and the "reasoning-budget-soft-ratio" are very welcome from my side.
I already use it with the Strix pentesting agent (https://github.com/usestrix/strix) and this is very helpful to "help" the model, in particular the MoE models Qwen3.6-35b and Ornith1.0-35b, to prevent overthinking and generate bullshit findings.

The "reasoning-budget-grace-tokens" should be changed and integrated into the "reasoning-budget" functionality per default or should be a boolean. Otherwise you could set "reasoning-budget-grace-tokens" to an infinity value and break the entire meaning of this. Or just delete it.