Question about OpenAILike usage

[antonsapt4]

Saw this LM-Studio Cookbook,

from phi.assistant import Assistant
from phi.llm.openai.like import OpenAILike
from phi.tools.duckduckgo import DuckDuckGo

assistant = Assistant(
    llm=OpenAILike(base_url="http://localhost:1234/v1"),
    tools=[DuckDuckGo()],
    show_tool_calls=True,
)
assistant.print_response("Whats happening in France? Summarize top stories with sources.", markdown=True)

Is it because LM-Studio OpenAI API provided is supporting tools?

Since I'm using colab wondering how OpenAILike works and tried with Groq endpoint llm=OpenAILike(base_url="https://api.groq.com/openai/v1", api_key="gsk_xxxxxx", model="mixtral-8x7b-32768"),

it said: tools is not supported with this model. Just want to get my head around it.

And kudos with phi-data it really make opensource model usable in very clear way without many workaround!

[ysolanky]

Hello! Groq currently does not support function calling. You can check out our Gorq cookbook and use Groq directly.

Since many providers use OpenAI API format, we use OpenAILike to access them.

[ashpreetbedi]

Thanks @antonsapt4, as @ysolanky pointed out Groq doesnt support tools yet. As soon as they do we should have that available as well :)

[antonsapt4]

Thanks @ysolanky @ashpreetbedi. Yes just read on groq website that the endpoint is not supported function calling.

After playing around, looks like only Ollama with openhermes model that can do tools calling, do you know what local model that can use the tools / function calling?

LM Studio even tho using the same OpenHermes (https://huggingface.co/TheBloke/OpenHermes-2.5-Mistral-7B-GGUF) looks like still cannot call tools using the code above.

Really appreciate if anyone had any experience using different model beside Ollama OpenHermes to have comparison.

[ashpreetbedi]

@antonsapt4 sadly currently we've only seen openhermes to be decent-ish at tool calling. We're still very early in the function calling journey, even so much that most LLM providers dont support it yet. We'll see this become mainstream in the next few months and you'll get more models/LLM providers supporting it

[ashpreetbedi]

@antonsapt4 try out hermes2 for function calling, it is very very good for a 7b model: https://github.com/phidatahq/phidata/tree/main/cookbook/hermes2