[Feature] Parallel tool execution

[quancore]

What feature would you like to see?

Is there any plan to make parallel tool execution feature for React etc?

Would you like to contribute?

• Yes, I'd like to help implement this.
• No, I just want to request it.

Additional Context

No response

[TomeHirata]

Hi, @quancore? Did you face a situation where LLM requests multiple tool calls in one turn, and each tool call took time? Would love to know more details to decide the priority!

[quancore]

Well many LLM modals are able to provide multiple tool calls at once. In our case we make data retrieval from different sources like web, API etc. and it makes those calls one by one @TomeHirata

[Ju-usc]

not sure if this is helpful, i approach parallel tool execution for ReAct by giving a tool to execute tools in parallel:

class ParallelToolCall:
    """Run multiple tool invocations concurrently (max 4 parallel threads).
    """

    def __init__(self, tools: Dict[str, Any], *, num_threads: int = 4) -> None:
        self.tools = tools
        self._num_threads = num_threads

    def __call__(self, calls: list[dict]) -> list[str]:
        """Calls must be dicts of the form {'tool': str, 'args': dict}."""
        if not calls:
            return []

        parallel = dspy.Parallel(num_threads=self._num_threads, provide_traceback=True)
        exec_pairs = [(self._invoke, (call,)) for call in calls]
        results = parallel(exec_pairs)
        return [str(result) for result in results]

    def _invoke(self, call: dict) -> Any:
        name = call.get("tool")
        args = call.get("args", {})
        tool = self.tools.get(name)
        if tool is None:
            return f"Unknown tool: {name}"

        try:
            return tool(**args)
        except Exception as error:
            logger.exception("Tool %s failed", name)
            return f"Tool error ({name}): {error}"

[quancore]

@chenmoneygithub Still up. There is a PR to give native tool call to React but it is still open? #8472

[TomeHirata]

@quancore Make sense, parallel tool call shouldn't be difficult.