Token Limiting Associated with Reasoning LLMs

[this-is-sebs]

Hello,
I have been using obsidian copilot for a while now. With the advent of reasoning models I am noticing that sometimes the obsidian copilot is not returning an output because of a token rate limit.

For example: I ask gpt-o3-mini-high a question that elicits a 2000 reasoning token output, but the output set in copilot is 1000 tokens. The obsidian copilot will obviously not return anything due to APIs not returning reasoning tokens. However, this often happens even when I increase the limit for tokens through copilot. If I ask a complex question it might exceed 12k-19k in reasoning tokens.

I was wondering if there was a specific way to limit the reasoning tokens before the AI then synthesizes those reasoning tokens into output tokens, through an api variable? Additionally, might it be possible to increase the LLM max output tokens beyond the maximum? This is just a problem I have stumbled into when using reasoning models with your platform. I know that some other reasoning models (DeepSeek...) may output even more verbose reasoning tokens beyond 19k. I wasn't sure if it was possible to separate the two, reasoning and output tokens. The costs associated with the reasoning tokens is also to be considered.

[logancyang]

Thanks for bringing this up. I think I'll first bump the max tokens limit in the setting, and then look into whether it's possible to treat reasoning tokens separately. Since reasoning model APIs have various different formats at the moment, it's unclear if it's possible.

[this-is-sebs]

Thank you. I noticed this was soon as the new update posted a couple of days ago and was not sure if a modification was made. I was noticing open router cap out at the completion tokens, as was indicated in copilot.

Also, are copilot beta keys still working? I had mine still plugged in and it was indicating active?

[logancyang]

@this-is-sebs you mean the alpha key or the one after you pay and generated yourself? Can you successfully use the copilot-plus-flash model?

[this-is-sebs]

@logancyang I was referring to the alpha key. I believe after I tried to use those models it returned an [Object object] output.

[logancyang]

@this-is-sebs that is a bug where it doesn't properly show the "unauthorized" error and still thinks it's "Active". Should be addressed by #1320