Document use of Mistral

[borisdayma]

It looks like you already support Mistral, though maybe missing sliding window attention.

Would be great to:

• add a section about it in https://github.com/google/maxtext#supported-open-models
• how to do inference (similar to gemma set-up instructions + use of decode.py)
• share the converted weights or add a conversion script

[borisdayma]

Looks like this is actually available: https://github.com/google/maxtext/blob/main/end_to_end/test_mistral.sh

The only thing I had to do was replace tokenizer.mistral with tokenizer.model (is it a typo or did you rename it in your bucket?).
Also I chose to convert the bfloat16 weights to float32 instead of float16 which I think could bring some imprecision.

[versae]

Can I ask what kind of TPU are you using for the test, @borisdayma? I do have available a v4-32 that I'd like to use to do continue pre-training on Llama2/Mistral 7B, but other frameworks seem sub-optimal so far to me.

[borisdayma]

It should work on a v3-8.
You can also try the decode.py function but for me it worked on the 7b models (gemma or mistral).

[rwitten]

Amazing @borisdayma! We don't actually official support Mistral (we do support Llama and Gemma) but we're thrilled things are working for you!

[borisdayma]

Yeah your inference test of mistral is correct. I compared with transformers output and was getting the same.

[borisdayma]

I'm closing this issue because Mistral seems to already work well after further testing.