Can you modify chunk idle time?

[tnguyen0000]

For inference routing, is there a way to modify/increase the chunk idle time? It seems to be hardcoded to 120 seconds.

I've already tried setting the timeout time higher, but it doesn't seem to affect the chunk idle time.

I'm asking as I'm using local inference (llama.cpp) on relatively weaker hardware and the 120s chunk idle time causes my agent/harness to stop and retry when reprocessing larger prompts (around 80-90k context reprocessing) especially when tool calling. The specific error I run into is: "error: response truncated: chunk idle timeout exceeded"

I've already looked at #866 but that issue was marked as not planned and then closed and I believe the codebase has been refactored since the issue was first raised(?).

Just for reference in case the docs are updated:

[johntmyers]

inference.local has been deprecated. Version 0.1.0 will be the first release without it. The release should go out before next week and the docs will show how to configure Providers for direct access to inference services.

[tnguyen0000]

Awesome, thanks for the response! I assume the chunk idle time issue is addressed in 0.1.0? And does the dev build (even if unstable) have the 0.1.0 features?