add support for OpenELM #63
Conversation
| public static let openelm270m4bit = ModelConfiguration( | ||
| id: "mlx-community/OpenELM-270M-Instruct" | ||
| ) { prompt in | ||
| "\(prompt)" |
There was a problem hiding this comment.
First off, phenomenal work! I tested it, and it seems to be doing completion just fine. Do you know how we should format it for instruction? Is it like the phi model at all?, or the other ones like the model above? I didn't see any special tokens for the instruct version thus far.
There was a problem hiding this comment.
Thanks very much Matthew. I've no idea what the chat template format should be and perhaps someone from Apple can comment.
There was a problem hiding this comment.
Ah, looks like I found something helpful here, just not sure how to translate to the code https://github.com/apple/corenet/blob/main/projects/openelm/instruction_tuning/openelm-instruct.yaml
There was a problem hiding this comment.
Oh that looks useful, let me grab the relevant content from the chat template and try it out. Thanks for looking into it.
There was a problem hiding this comment.
@MatthewWaller I tried the following template without luck.
"<|system|>\nYou are a helpful assistant<|end|>\n<|user|>(prompt)<|end|>\n<|assistant|>"
There was a problem hiding this comment.
Using the default template for the Llama tokenizer, <s>[INST]\(prompt)[/INST] seems to work but I'll leave it to someone who knows better.
There was a problem hiding this comment.
Yeah, overall this is working well for completion. I thing for chat there are some things we can do after this PR: there is a bug in swift transformers that just got fixed in main pertaining to an encoding special tokens. AND the config isn’t setup to recognize tokens like “<|user|>” and such, so we would need to adjust that too eventually.
|
@smdesai could you run the swift formatting? |
|
@awni It's run. |
@awni Here's the PR