Nonsense output on MPT-7b models

[georgezlei]

I tested 3 models in Oobabooga on my M1 Macbook Pro:

• MPT-7b-chat
• MPT-7b-storywriter
• Zephyr-7b-beta

Among them, Zephyr-7b works well. But the 2 MPT-7b models won't generate anything readable. Below is an example:

Prompt:

Once upon a time there was a poor blacksmith. He discovered a novel type of metal from a miner in the Ur mountains. Upon forging a blade from it he found

Output:

Once upon a time there was a poor blacksmith. He discovered a novel type of metal from a miner in the Ur mountains. Upon forging a blade from it he found found out that this new old once upUrUrUrUrurrrr rRrrrrRRUR RR UR ur uuUuuUU UuuuruowOWoWwwoWWwwwww www.."." "Minor" Minors, andandadadadAd Advertising advertising ad advertising advertiser advertisement advertisinigigigigiGi Gi G iiijijiji jjjjJJ JJJ J Ji K kkkeleLeLLEl LLLll lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllwww : IIII II

This result is very different from my other tests of MPT-7b in Colab. Are there any settings I can try?