NodeLLM 1.17 adds opt-in parallel tool execution #75
eshaiju
announced in
Announcements
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
When a model returns several independent tool calls in the same turn, NodeLLM has always run them one at a time. That's safe by default, but wastes time when the calls don't depend on each other — three weather lookups for three cities, say. Enable
toolConcurrencyto run them in parallel instead:Works the same way in
stream()and Agent mode — no changes needed to how your tools are defined.Upgrade:
npm install @node-llm/core@1.17.0All reactions