Support ggup format?

[hiqsociety]

llama.cpp uses ggup now.

can provide an eta on when this feature will be available?
thx in advance.

[mudler]

you mean gguf? gguf it's already supported

[y4my4my4m]

@mudler i havent' had any luck using GGUF. Are there special steps to take?
The backend should be llama and not llama-stable right? Apart from that, anything else? Can we use CUDA? Thread config, etc?

[mudler]

What's the error you have? gguf files should work and are tested automatically by the CI.