fix mistral prompt assembly

[hamelsmu]

Summary

Similar to the llama prompts (#952 ) I found some inconsistencies with the mistral prompts. There are still some small differences with spaces but this makes the prompt conform more to the official prompt.

Key differences between Mistral and Llama that I fixed:

• No BOS token that separates conversations, only EOS token
• There is no official guidance on how to handle the system message, so I left this consistent with llama

I Also found that previously, we weren't correctly handling the case where there was no system message and were not adding the [INST] at the beginning.

I also took the opportunity to clean up the tests since I'm adding more.

The best way to understand this PR is to look at the test cases.

References

https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1
and
https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2

The best way to "see" the prompt template most people are likely to use is to use HF Chat template:

from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2")

chat = [{"role": "user", "content": "abc"}, 
        {"role": "assistant", "content": "ipsum"}, 
        {"role": "user", "content": "123"}, 
        {"role": "assistant", "content": "sit"}]

ids = tokenizer.apply_chat_template(chat)
print(tokenizer.decode(ids))

Which results in

<s> [INST] abc [/INST]ipsum</s> [INST] 123 [/INST]sit</s>

[hamelsmu]

cc: @tokestermw @casper-hansen @winglian

[casper-hansen]

I’m not too happy seeing so many of our methods for promoting have errors. I wonder if this fixes any performance gap?

[hamelsmu]

I’m not too happy seeing so many of our methods for promoting have errors. I wonder if this fixes any performance gap?

@casper-hansen I have the same question. I think its one of the most important questions to answer. I'm going to be doing a systematic study of this. LMK if you want to help