StableCode Fails During Inference (stabilityai/stablecode-completion-alpha-3b-4k)

[JustinMeans]

stabilityai/stablecode-completion-alpha-3b-4k fails at the assert here but converts successfully using:
python -m exporters.coreml --model=stabilityai/stablecode-completion-alpha-3b-4k exported/

It's a gpt_neox model with GPTNeoXForCausalLM architecture.

When removing the assert and re-running, the SwiftChat app is able to load and run the model, but generates gibberish.

I have the converted model currently uploading to HF but it's going to take a bit to complete.

[pcuenca]

Thanks for the report, @JustinMeans!

The default task in exporters is feature-extraction: https://github.com/huggingface/exporters/blob/80d37d8004e31a9cbd00ce4bc5ef261c175f62ce/src/exporters/coreml/__main__.py#L83

Could you please try to convert adding --feature text-generation to your command line?

(We should probably use different defaults in exporters based on the model type).

[pcuenca]

Closing for now, feel free to reopen if appropriate!