Skip to content

How to reject the tool calling while using User-Controlled Tool Execution #4871

@ingbyr

Description

@ingbyr

How to reject certain tool calling requests in the process of using User-Controlled Tool Execution.

import lombok.RequiredArgsConstructor;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.messages.AssistantMessage;
import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.ai.chat.model.Generation;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.ai.model.tool.ToolCallingChatOptions;
import org.springframework.ai.model.tool.ToolCallingManager;
import org.springframework.ai.model.tool.ToolExecutionResult;
import org.springframework.ai.tool.ToolCallbackProvider;
import org.springframework.stereotype.Component;

import java.util.Scanner;

@Component
@RequiredArgsConstructor
public class ToolCallingAgent {

    private final ChatClient.Builder builder;
    private final ToolCallbackProvider mcpTools;

    public void run() {
        ToolCallingManager toolCallingManager = ToolCallingManager.builder().build();
        ToolCallingChatOptions chatOptions = ToolCallingChatOptions.builder()
                .temperature(0.1)
                .toolCallbacks(mcpTools.getToolCallbacks())
                .internalToolExecutionEnabled(false)
                .build();
        Prompt prompt = new Prompt("""
                Execute tool calls when necessary. If a response does not contain tool call results, 
                skip this tool call and continue with subsequent tool call tasks.
                """, chatOptions);

        ChatClient chatClient = builder.build();
        ChatResponse chatResponse = chatClient
                .prompt(prompt)
                .user("What directories are under the 'code' directory, and what is the purpose of each directory?")
                .call()
                .chatResponse();
        System.out.print(chatResponse.getResult().getOutput().getText());

        Scanner scanner = new Scanner(System.in);
        while (chatResponse.hasToolCalls()) {
            for (Generation generation : chatResponse.getResults()) {
                for (AssistantMessage.ToolCall toolCall : generation.getOutput().getToolCalls()) {
                    System.out.printf("%s (tool call %s)\n", generation.getOutput().getText(), toolCall.toString());
                }
            }
            System.out.print("Allow to execute tool calling ?(y/n): ");
            String confirm = scanner.nextLine().trim().toLowerCase();
            if ("y".equals(confirm)) {
                ToolExecutionResult toolExecutionResult = toolCallingManager.executeToolCalls(prompt, chatResponse);
                prompt = new Prompt(toolExecutionResult.conversationHistory(), chatOptions);
                chatResponse = chatClient.prompt(prompt).call().chatResponse();
                System.out.println(chatResponse.getResult().getOutput().getText());
            } else {
                // TODO Reject tool calling
                // Expecting some methed like `toolCallingManager.rejectToolCalls(prompt, chatResponse);`
                System.out.println("User cancel the tool calling operation");
            }
        }
        System.out.println(chatResponse.getResult().getOutput().getText());
        System.out.println("Finished");
    }
}

The output logs of LLM are currently similar to the content below: when tool calling is not executed, the system repeatedly requests the execution of the same tool calling.

 (tool call ToolCall[id=019a7c85fcdec2d390a2ba15d21b7534, type=function, name=list_directory, arguments={"path": "/path1"}])
Allow to execute tool calling ?(y/n): y

 (tool call ToolCall[id=019a7c86356f17e95d977e986f462643, type=function, name=list_directory, arguments={"path": "/path2"}])
Allow to execute tool calling ?(y/n): y

 (tool call ToolCall[id=019a7c866b12e22800aa8de2a659a339, type=function, name=list_directory, arguments={"path": "/path3"}])
Allow to execute tool calling ?(y/n): n
User cancel the tool calling operation

 (tool call ToolCall[id=019a7c866b12e22800aa8de2a659a339, type=function, name=list_directory, arguments={"path": "/path3"}])
Allow to execute tool calling ?(y/n): n
User cancel the tool calling operation

 (tool call ToolCall[id=019a7c866b12e22800aa8de2a659a339, type=function, name=list_directory, arguments={"path": "/path3"}])
Allow to execute tool calling ?(y/n): n
User cancel the tool calling operation

Is there a possibility to add a method similar to toolCallingManager.rejectToolCalls(prompt, chatResponse) for rejecting tool calls?

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions