Skip to content

Investigate running Tools in Parallel in the ToolInvoker #9008

Open
@sjrl

Description

@sjrl

Based on the feedback from @d-kleine in the discussion here it would be interesting to explore whether we could run the tools in ToolInvoker in a parallel way.

This is relevant since it's possible for an Agent to request multiple tools be called at once before continuing the generation.

Perhaps it's possible we could implement this using Threading when using Pipeline and if we add an Agent based on AsyncPipeline maybe it's possible to add a run_async method to ToolInvoker.

Metadata

Metadata

Assignees

Labels

P1High priority, add to the next sprint

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions