Mario System Prompt not working with Mistral Model

[ghost]

In this example: https://github.com/jmorganca/ollama/blob/main/examples/mario/readme.md

I can successfully create a new model with mistral, however it seems to ignore the system prompt. I tried various system prompts but seems to revert back to Mistral.

Here is my results:

ollama run MARIO

who r u?

I am Mistral...

[BruceMacD]

Looks like mistral doesn't have a system prompt in its default template:

ollama run mistral
>>> /show modelfile
# Modelfile generated by "ollama show"
# To build a new Modelfile based on this one, replace the FROM line with:
# FROM mistral:latest

FROM /Users/bruce/.ollama/models/blobs/sha256:6ae28029995007a3ee8d0b8556d50f3b59b831074cf19c84de87acf51fb54054
TEMPLATE """[INST] {{ .Prompt }} [/INST]
"""
SYSTEM """"""

Try this Modelfile instead for the Mario example:

FROM mistral
PARAMETER temperature 1
TEMPLATE """[INST] <<SYS>>You are Mario from Super Mario Bros, acting as an assistant.<</SYS>>

{{ .Prompt }} [/INST] """

I haven't tested this template out much, but it may do the trick. Iterate as needed.

[cjbottaro]

Adding that TEMPLATE directive to the Modelfile worked for me.

Btw, where can I learn more about that syntax like <<SYS>> and [INST]? I'm new to LLMs and such.

[65a]

@cjbottaro these sorts of things are usually model-specific and come from how it was trained. You can usually find them in the huggingface modelcard for the specific model, or the model it was derived from. For Mistral: https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1 the instruction format describes [INST]. They often will work with a variety of prompt styles, but you can imagine they work best with prompt formats they saw during training best.

[s-kostyaev]

Can we fix default template for mistral? I can't find default templates for models to provide PR.

[jmorganca]

This should be fixed now. Thanks for creating an issue and feel free to re-open if you see this again!