[Question] Is GGUF model package format supported with quantized models?

[qdrddr]

❓ General Questions

Hi, can I use existing gguf model package format of quantized models such as this one?
https://huggingface.co/MaziyarPanahi/Mixtral-8x22B-Instruct-v0.1-GGUF

If not, can I convert it to MLC?

[Hzfengsy]

GGUF is not supported with MLC-LLM. You could convert weight from the original model. See https://llm.mlc.ai/docs/compilation/convert_weights.html

[qdrddr]

Sorry for the stupid question, but aren't Hugging Face and GGUF two different formats?
I am specifically interested in GGUF and the manual you provided for HF format. Or are both supported by the converter? @Hzfengsy

[Hzfengsy]

Yes Huggingface and GGUF are different formats. GGUF is not supported by the converter now