Error: invalid file magic when creating an xs model

[eramax]

Hi,
I tried to create a new model using this gguf file chat-67b-xs.gguf but i didn't work and gave me this output.
I think the xs models is not being supported yet by ollama, but it is working fine the same file using llama.cpp ~/dev/llama.cpp/main --color --instruct -ngl 100 -m deepseek-chat-67b-xs.gguf

➜  models ollama create deepseek-chat-67b-xs
transferring model data
creating model layer
Error: invalid file magic

Modelfile

FROM ./deepseek-chat-67b-xs.gguf

TEMPLATE """{{ .System }}
### Instruction:
{{ .Prompt }}
### Response:"""

PARAMETER stop "<s>"
PARAMETER stop "### Instruction:"
PARAMETER stop "### Response:"

PARAMETER num_ctx 2048
PARAMETER temperature 0.3
#PARAMETER top_k 40
#PARAMETER top_p 0.8
#PARAMETER num_predict 1024

SYSTEM """You are an AI programming assistant"""

[eramax]

@jmorganca , does this a problem in my side only or IQ xs models aren't supported yet?

[virt-god]

Someone managed to do it.
Model: https://ollama.com/impactframes/mistral_alpha_xs
Post: https://www.reddit.com/r/ollama/comments/1aozwms/mistral_alpha_xs_knut_j%C3%A4gersbergs_2bit_imatrix/

Also since it seems to be supported will IQ3_XXS support be added?

I have also been trying to do this but with no success I even compiled version 0.1.25 and 0.1.21 as stated in the post.
Maybe there is something wrong with the arch PKGBUILD?

Edit:
Tried it with the official install script didn't work.

[bmizerany]

I'm unable to reproduce with the latest version of Ollama. I'm going to close this for now, but please reopen if the issues persists.

My output using your provided Modelfile and the gguf model:

; ls
Modelfile			deepseek-chat-67b-xs.gguf
; ollama create deepseek-chat-67b-xs
transferring model data 
creating model layer 
creating template layer 
creating system layer 
creating parameters layer 
creating config layer 
using already created layer sha256:cdbc2eeea6d46a3985b151ea03bdc16b83883d2e5de16d808c3c4b54779e0c40 
writing layer sha256:071ef283861a9a36eca353dd6128b9f3b114a09d7888db6fe0b782cc46ecc08c 
writing layer sha256:1a34d9fe82c00072c2b2657e8c0df1c506e2c6c513e0bff4020dea81702d5dc2 
writing layer sha256:72417c131801d602397eeb0e9d45bdacafce5437442154f5b3b52dc7c31c003e 
writing layer sha256:ea55261939a21fd7611604c99f46745840fec2830440ac4faef0856b98ffd4bb 
writing manifest 
success 
; ollama run deepseek-chat-67b-xs       
>>> hello
Hello, how can I assist you today?

>>> Send a message (/? for help)

[oldmanjk]

It seems only certain IQ quants are supported? Could we get the rest supported or can a list of the supported ones be posted prominently on the main readme? Kind of annoying to do all the work only to find out it's not supported

[bmizerany]

I can see how that would be frustrating, @oldmanjk. There is more discussion about this going on in #3622. We'll keep investigating.

[h34tnet]

i just stumbled upon that error. first i downloaded [Meta-Llama-3-70B-Instruct-IQ2_XS.gguf](https://huggingface.co/lmstudio-community/Meta-Llama-3-70B-Instruct-GGUF/blob/main/Meta-Llama-3-70B-Instruct-IQ2_XS.gguf) and converted it to llama3-70b-iq2 by using FROM path/to/model.gguf - this worked. then i tried to recreate the model (to change the system prompt) by creating a new model file using FROM llama3-70b-iq2 and this failed:

ollama create llama3-70b-iq2-kotlin -f .\llama3-70b-iq2-kotlin
transferring model data
creating model layer
Error: invalid file magic

[twobombs]

same here on the IQ3_M file - will fetch externally at runtime, but would be great if integrated at build time