llama-ui: allow me to enable/disable reasoning on-demand #29611
jerkstorecaller
started this conversation in
Ideas
Replies: 1 comment 1 reply
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment

Uh oh!
There was an error while loading. Please reload this page.
I have llama-ui running Qwen 27B all the time. It likes to think a lot, which is good for certain types of tasks. But sometimes I just want a quick answer where reasoning is less important, such as language translation for a casual chat.
I don't see an option in llama-ui to disable reasoning for a specific chat convo.
I did see that I can let it start answering, it begins to reason, then a Skip Reasoning button shows up, and if I click it, it seems to terminate thinking and continue outputting the answer. But I'm not sure if there's accuracy consequences of having a half-finished thought at the beginning of the LLM reply.
I think llama-ui would benefit from a setting to specify reasoning levels. For example, when clicking the + in the lower left of the chat bar, have a Reasoning Level submenu, which lets you select between Reasoning Off, Low, Medium, High, XHigh. If that's too complicated/model-dependent, then what about a Reasoning Enabled entry that is checkable?
All reactions