Improve AI provider reliability: timeouts, retries, streaming errors, and fallback handling #12461
michaelmanly
started this conversation in
Feature Requests
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I noticed a few AI provider/runtime paths where production reliability behavior may currently depend on broad defaults or single-route execution:
packages/fetch/src/getAgentOptions.tsTIMEOUTappears to default to7200secondscore/util/withExponentialBackoff.ts429or overload eventscore/util/withExponentialBackoff.ts"malformed json"core/llm/llms/OpenAI.ts_streamResponsesappears to catch malformed streaming chunk parse errors and ignore themcore/llm/streamChat.tsmodel.streamChat(...)and rethrow on failureWould maintainers be open to a small PR to improve one of these reliability paths?
Possible small PR options:
The goal would be optional production reliability improvements without changing default behavior aggressively or adding a required dependency.
Beta Was this translation helpful? Give feedback.
All reactions