Issue with LLMs Not Following JSON Format Instructions for Avatar Animations

[XredaX]

Problem

I found a problem with some LLMs in our system. I've added instructions in the backend to make the LLM respond in JSON format (needed for avatar animations), but several models only respond in plain text, ignoring these format instructions.

Models Tested

1. Works: Gemini (but it's external, not in Ollama, and we need to stay internal)
2. Doesn't Work: Gemma:2b, Gemma:7b, and Mistral (all of them ignore JSON format instructions)

Request for Help

If you have suggestions for Ollama-compatible models that reliably follow format instructions and can output structured JSON consistently, please recommend them.
Any specific prompt techniques or configuration settings that help?

As you can see, when using gemma:2b as the model, the response is plain text. However, when switching to the Avatar Backend Pipeline model, which is powered by Gemini, the response is structured as JSON. This is exactly what we want, as it allows for better integration with animations and avatar interactions.

[pr-elhajji]

Hi,

I think Gemini offers compatibility with OpenAI. You might want to check out this link for more details:

https://ai.google.dev/gemini-api/docs/openai?hl=fr

Let me know what you think!

[XredaX]

Thank you very much for your help,
That’s great! I think this is awesome, and I’m going to try it.

ll let you know how it goes!