Broken Tool/Function implementation

[JpEncausse]

Context

I try to figure out of to make LLM with Tool works.

• Targeting Phi-4 Mini
• Use case is Home Automation

I implement an OpenSource Node-RED Framework for

• Azure GPT4o
• Ollama
• Foundry Local

Why LLM Tools Sucks ?

Here is a details Reddit explanation of what I try to do. A very simple RAG Query. A real life prompt that return a JSON or delegate to a Tool

• It works on GPT4o (Cloud + JS Library)
• It fails or weak on Ollama / Foundry

I tested
Qwen3:8b, Qwen3:30b-a3b, Qwen2.5-coder, GPT4o, Devstral, Phi4-mini, Command-r7b, Mistral-nemo, Llama4:scout
MHKetbi/Unsloth-Phi-4-mini-instruct

Foundry Local

Question asked on Discord, I don't understand the mess between tools and function. And it seems implementation is really really buggy.

According to the documentation

• functions is the way to declare tools,
• tools is also available on Foundry Locals but Microsoft doc talk about tools and OpenAI ChatGPT use tools.
• function_call seems to be new in request (not response) ?

If I work around to force a tool mechanic at the 2nd request (close to ChatGPT 4o) I got the following error :
$.tool_calls[0].function.arguments | LineNumber: 0 | BytePositionInLine: 150

It also seems there is a mismatch between tool_calls and toolCalls

Question

• Is it a bug or a feature ;-)
• Can you explain Tools vs Function
• Can you share a REST request : Prompt + Function then REST request with Functions's content

Step Back

Why Local LLM are some week at feeding Tools ? For instance I only have 1 tool and only ask for 2 arguments for a search engine. So it's really basic no ? Half of the time "semantic sentence" argument is bad. It could be just a word (that's very semantic !) or rephrase the query "search bla bla ..." (it could remove the "search" part) we already querying a search engine.

I probably do something wrong in my prompt ? GPT4o and Qwen handle it correctly but other are very bad.

AB#66028

[MaanavD]

Hey there @JpEncausse , we have this: https://aka.ms/FLAgentPrp where you can sign up for agent mode testing!

Otherwise I'm not sure what 'functions' are or what your code looks like for that! The link should be a good starting point :)

[JpEncausse]

Hello, I'm MVP already sign-in but didn't understand Foundry Local and Foundry Local Agent were 2 different projects !

About

If I work around to force a tool mechanic at the 2nd request (close to ChatGPT 4o) I got the following error :
$.tool_calls[0].function.arguments | LineNumber: 0 | BytePositionInLine: 150

The issue was it required a String of the json for arguments not the json itself. But I get other bugs implementing the rest.
And anyways

• I think it do not handle Tools correctly
• I still don't understand tools vs function in Microsoft documentation

FYI my project look like that
https://github.com/jpEncausse/nodered.encausse.net

The video:
https://www.youtube.com/watch?v=jE_VykxWFv8

Basicly it's Node-RED nodes to handle LLM & SLM and so visually chain them

[natke]

Thank you for providing this detailed feedback @JpEncausse. We are investigating