Update assistant timeout to accomodate longer response time.

[petrpan26]

Relating to:
#174

An improvement idea is that we can have streaming input check for those api that support this (e.g GPT). This way we know that it's still generating output rather than a network problem.

[pmeier]

Thanks for the fix @petrpan26!

An improvement idea is that we can have streaming input check for those api that support this (e.g GPT).

Unfortunately, this is not really possible or will be a lot harder than you might think. I would very much like to have this feature and have played around with it in the past.

This is so hard, because we receive the response on the worker, but consume it on the API side. In between them is our result storage. Meaning, we would need to somehow implement a streaming approach for the results and finally also implement streaming on our own API.

If you want to have a go at this, be my guest. But be warned that this likely could turn really ugly.