Auto-adjust reasoning effort based on task difficulty #5254
miglaginf-tech
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Today the effort level (low/high/max) is chosen manually by the user, with no guidance from the agent. In my real usage, the cost is almost entirely concentrated in reasoning tokens, and a large portion is "over-spent" on tasks that don't need it.
My proposal: before starting, the agent should recommend the level based on the nature of the task (e.g., technical OCR + complex document with figures and formulas → high/max; summary, formatting, or ordinary conversation → low), and afterwards confirm whether the level used was appropriate (or whether it was too low / overkill). Ideally, it could auto-adjust in subsequent steps within the same session, since the technical mechanism for changing effort between steps already exists.
Reference case: simple tasks measured ~60% of output tokens as reasoning, when low would have been enough. Well calibrated, this would noticeably cut cost without affecting quality on hard tasks.
All reactions