-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Open
Description
Hi team 👋
I have a suggestion for using ChatClient API.
Expected Behavior
I believe it's going to be great if the ChatClient API support timeout configuration at both global and per-request levels. Here's what I envision:
// Option 1: Per-request timeout
ChatResponse response = chatClient
.prompt("Hello, world!")
.options(chatOptions)
.timeout(Duration.ofSeconds(30))
.call();
// Option 2: Builder-level default timeout
ChatClient client = ChatClient.builder(chatModel)
.defaultTimeout(Duration.ofSeconds(30))
.build();
// Option 3: ChatOptions integration
ChatOptions options = ChatOptions.builder()
.temperature(0.7)
.timeout(Duration.ofSeconds(10))
.build();
If you agree with this feature request, I'll try to contribute it.
Thank you.
dev-jonghoonpark, injae-kim, macro161, cherryfeng1026, Romancha and 2 more