You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It would also be useful to provide a ContextTooLargeError, with error translation for anthropic, google, and openai - and compatible.
Why
I am primarily interested in being able to continue conversations in Crush which, for whatever reason, accidentally overflow:
The above is what happens if you overflow + try and compact. To resolve this issue elegantly, we need to understand the error from the API within crush. IMO, the best way tot do that is through error translation. Then crush can recover as it sees fit (perhaps dropping last message which caused the overflow and summarizing).
// errors.go additiontypeContextTooLargeErrorstruct {
UsedTokensint// Actual tokens in request (if parseable)MaxTokensint// Model's limit (if parseable)Modelstring// Model that rejected the requestCauseerror// Underlying ProviderError
}
// Each provider's toProviderErr() would be modified:functoProviderErr(errerror) error {
provErr:=// existing logic...// New: detect context overflow and wrapifctxErr:=parseContextError(provErr); ctxErr!=nil {
returnctxErr
}
returnprovErr
}
This would require writing translators for at least anthropic (and compat), Gemini and OpenAI (and compat).
I would be happy to create these! Please let me know.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
It would also be useful to provide a ContextTooLargeError, with error translation for anthropic, google, and openai - and compatible.
Why
I am primarily interested in being able to continue conversations in Crush which, for whatever reason, accidentally overflow:
The above is what happens if you overflow + try and compact. To resolve this issue elegantly, we need to understand the error from the API within crush. IMO, the best way tot do that is through error translation. Then crush can recover as it sees fit (perhaps dropping last message which caused the overflow and summarizing).
This would require writing translators for at least anthropic (and compat), Gemini and OpenAI (and compat).
I would be happy to create these! Please let me know.
Beta Was this translation helpful? Give feedback.
All reactions