Skip to content

Conversation

@NatashaTheRobot
Copy link
Contributor

Many models take a much longer time to generate a response - e.g. for reasoning models, processing big PDFs or video files, etc. The 60 second timeout is not enough, so I increased it to 400. Feel free to edit this in a way that the user can set the configuration if that makes more sense.

@lzell
Copy link
Owner

lzell commented Mar 27, 2025

Thank you! I have to think about this one a little bit. If there are existing callers that depend on the default timeout of 60 (say catching and popping UI), then this will change the UX for their end user. I think it's likely OK.

I do have some calls in the sdk that expose a secondsToWait param. Perhaps there is a principled way to apply that to all calls. Maybe by opening up a param like you did here, but default it to 60, and then modify all the XYZService protocol definitions to accept a secondsToWait argument. This will cause compilation errors on the next build for everyone that upgrades, but it forces the issue of them putting in a secondsToWait that makes sense for their use case

@NatashaTheRobot
Copy link
Contributor Author

NatashaTheRobot commented Mar 27, 2025

@lzell - agreed. I think it's better to have it as some kind of model configuration. Wasn't sure how to implement it. For some models it makes sense to have the default, but others now might need longer, so it's nice to have control over that as a developer on a per model basis to manage expectations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants