Is GGUF support coming anytime soon ?

[PedzacyKapec]

Feature request

Is GGUF support coming anytime soon ? Nobody seems to be doing GGML anymore as the project was phased out about a month ago.

Motivation

Yes

Your contribution

What?

[manyoso]

Yes, it is the next highest priority goal after getting a nightly build and I'll be working on it this coming week.

[TheCopacabanaMan]

Keep up your good word, manyoso!
You, and all your team!

[JamshedQurbonboev]

How progress with GGUF support is going?

[cebtenzzre]

GGUF support is currently available in this pre-release, which is available for beta testing:
https://github.com/nomic-ai/gpt4all/releases/tag/v2.5.0-pre1

[PedzacyKapec]

GGUF support is currently available in this pre-release, which is available for beta testing: https://github.com/nomic-ai/gpt4all/releases/tag/v2.5.0-pre1

GGUF "only" ?
What do you mean ? No GGML support ?
WHy cant both be supported ?

[cebtenzzre]

WHy cant both be supported ?

It's a lot of effort to maintain support for older file formats, since the current version of llama.cpp only supports GGUF. GGUF is intended to be a long-term solution that can easily be extended, so it should be the last new format for a long time.

There is a script in llama.cpp that you can use to convert LLaMA models to the new format, called 'convert-llama-ggml-to-gguf.py'. Many of the models previously available in GPT4All are available for download in the new format, and many GGML models quantized by TheBloke have GGUF variants now as well.