-
Notifications
You must be signed in to change notification settings - Fork 2k
Open
Description
How to reject certain tool calling requests in the process of using User-Controlled Tool Execution.
import lombok.RequiredArgsConstructor;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.messages.AssistantMessage;
import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.ai.chat.model.Generation;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.ai.model.tool.ToolCallingChatOptions;
import org.springframework.ai.model.tool.ToolCallingManager;
import org.springframework.ai.model.tool.ToolExecutionResult;
import org.springframework.ai.tool.ToolCallbackProvider;
import org.springframework.stereotype.Component;
import java.util.Scanner;
@Component
@RequiredArgsConstructor
public class ToolCallingAgent {
private final ChatClient.Builder builder;
private final ToolCallbackProvider mcpTools;
public void run() {
ToolCallingManager toolCallingManager = ToolCallingManager.builder().build();
ToolCallingChatOptions chatOptions = ToolCallingChatOptions.builder()
.temperature(0.1)
.toolCallbacks(mcpTools.getToolCallbacks())
.internalToolExecutionEnabled(false)
.build();
Prompt prompt = new Prompt("""
Execute tool calls when necessary. If a response does not contain tool call results,
skip this tool call and continue with subsequent tool call tasks.
""", chatOptions);
ChatClient chatClient = builder.build();
ChatResponse chatResponse = chatClient
.prompt(prompt)
.user("What directories are under the 'code' directory, and what is the purpose of each directory?")
.call()
.chatResponse();
System.out.print(chatResponse.getResult().getOutput().getText());
Scanner scanner = new Scanner(System.in);
while (chatResponse.hasToolCalls()) {
for (Generation generation : chatResponse.getResults()) {
for (AssistantMessage.ToolCall toolCall : generation.getOutput().getToolCalls()) {
System.out.printf("%s (tool call %s)\n", generation.getOutput().getText(), toolCall.toString());
}
}
System.out.print("Allow to execute tool calling ?(y/n): ");
String confirm = scanner.nextLine().trim().toLowerCase();
if ("y".equals(confirm)) {
ToolExecutionResult toolExecutionResult = toolCallingManager.executeToolCalls(prompt, chatResponse);
prompt = new Prompt(toolExecutionResult.conversationHistory(), chatOptions);
chatResponse = chatClient.prompt(prompt).call().chatResponse();
System.out.println(chatResponse.getResult().getOutput().getText());
} else {
// TODO Reject tool calling
// Expecting some methed like `toolCallingManager.rejectToolCalls(prompt, chatResponse);`
System.out.println("User cancel the tool calling operation");
}
}
System.out.println(chatResponse.getResult().getOutput().getText());
System.out.println("Finished");
}
}The output logs of LLM are currently similar to the content below: when tool calling is not executed, the system repeatedly requests the execution of the same tool calling.
(tool call ToolCall[id=019a7c85fcdec2d390a2ba15d21b7534, type=function, name=list_directory, arguments={"path": "/path1"}])
Allow to execute tool calling ?(y/n): y
(tool call ToolCall[id=019a7c86356f17e95d977e986f462643, type=function, name=list_directory, arguments={"path": "/path2"}])
Allow to execute tool calling ?(y/n): y
(tool call ToolCall[id=019a7c866b12e22800aa8de2a659a339, type=function, name=list_directory, arguments={"path": "/path3"}])
Allow to execute tool calling ?(y/n): n
User cancel the tool calling operation
(tool call ToolCall[id=019a7c866b12e22800aa8de2a659a339, type=function, name=list_directory, arguments={"path": "/path3"}])
Allow to execute tool calling ?(y/n): n
User cancel the tool calling operation
(tool call ToolCall[id=019a7c866b12e22800aa8de2a659a339, type=function, name=list_directory, arguments={"path": "/path3"}])
Allow to execute tool calling ?(y/n): n
User cancel the tool calling operation
Is there a possibility to add a method similar to toolCallingManager.rejectToolCalls(prompt, chatResponse) for rejecting tool calls?