Problem when launching privateGPT

[PierreVannier]

Hi

I've done all the necessary steps to have a llama.cpp (vigogne) model and correctly ingested documents (pdf, docx, ppt) but when I launch privateGPT I have this error :

(base) MacBook-Pro-de-Pierre:privateGPT pierrevannier$ python privateGPT.py 
Using embedded DuckDB with persistence: data will be stored in: db
llama.cpp: loading model from models/ggml-model-q4_0.bin
error loading model: unknown (magic, version) combination: 67676a74, 00000003; is this really a GGML file?
llama_init_from_file: failed to load model
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Users/pierrevannier/privateGPT/privateGPT.py:75 in <module>                                     │
│                                                                                                  │
│   72                                                                                             │
│   73                                                                                             │
│   74 if __name__ == "__main__":                                                                  │
│ ❱ 75 │   main()                                                                                  │
│   76                                                                                             │
│                                                                                                  │
│ /Users/pierrevannier/privateGPT/privateGPT.py:33 in main                                         │
│                                                                                                  │
│   30 │   # Prepare the LLM                                                                       │
│   31 │   match model_type:                                                                       │
│   32 │   │   case "LlamaCpp":                                                                    │
│ ❱ 33 │   │   │   llm = LlamaCpp(model_path=model_path, n_ctx=model_n_ctx, callbacks=callbacks    │
│   34 │   │   case "GPT4All":                                                                     │
│   35 │   │   │   llm = GPT4All(model=model_path, n_ctx=model_n_ctx, backend='gptj', callbacks    │
│   36 │   │   case _default:                                                                      │
│                                                                                                  │
│ /Users/pierrevannier/privateGPT/pydantic/main.py:341 in pydantic.main.BaseModel.__init__         │
│                                                                                                  │
│ [Errno 2] No such file or directory: '/Users/pierrevannier/privateGPT/pydantic/main.py'          │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ValidationError: 1 validation error for LlamaCpp
__root__
  Could not load Llama model from path: models/ggml-model-q4_0.bin. Received error  (type=value_error)
(base) MacBook-Pro-de-Pierre:privateGPT pierrevannier$ 

I installed the latest version of llama.cpp, vigogne and bin running on a mac book pro M1.
Any clue ?

[lanalancia]

A model is not downloading automatically. by default it tries to load models/ggml-gpt4all-j-v1.3-groovy.bin , which is missing

To fix:

1. download https://gpt4all.io/models/ggml-gpt4all-j-v1.3-groovy.bin (same link in a repo readme.md file)
2. create a folder models
3. put your model in it

You can edit path and model name by editing .env

[pseudotensor]

@PierreVannier , with the latest langchain and gpt4all, it downloads the file automatically like hugginface would. It's probably easier to use that way.

You can try at: https://github.com/h2oai/h2ogpt . See updated instructions at: https://github.com/h2oai/h2ogpt/blob/main/FAQ.md#CPU

PrivateGPT can upgrade as well following h2oGPT code.

[PierreVannier]

@lanalancia @pseudotensor I want to be able to use a french model, that's why I use LLama CPP and Vigogne.
Here it says it fails to load the model whereas the model is a .bin Llama.cpp, converted, quantized etc...
This is odd since with previous version it worked.

[sjssmits]

I have the same error and problem. Can't figure out how to solve it, using the alpaca-lora-7B from Huggingface.

[Guillaume-Fgt]

Hi, the latest version of llama-cpp-python is 0.1.55. Do you have this version installed? pip list to show the list of your packages installed.

If not:
pip install --force-reinstall --ignore-installed --no-cache-dir llama-cpp-python==0.1.55

Then, you need to use a vigogne model using the latest ggml version: this one for example.

THE FILES IN MAIN BRANCH REQUIRES LATEST LLAMA.CPP (May 19th 2023 - commit 2d5db48)!

llama.cpp recently made another breaking change to its quantisation methods - ggerganov/llama.cpp#1508

I have quantised the GGML files in this repo with the latest version. Therefore you will require llama.cpp compiled on May 19th or later (commit 2d5db48 or later) to use them.

For me, it is working with Vigogne-Instruct-13B.ggmlv3.q4_0.bin. But I am on windows, so can't say 100% it will on your machine.

[PierreVannier]

Hi, the latest version of llama-cpp-python is 0.1.55. Do you have this version installed? pip list to show the list of your packages installed.

If not: pip install --force-reinstall --ignore-installed --no-cache-dir llama-cpp-python==0.1.55

Then, you need to use a vigogne model using the latest ggml version: this one for example.

THE FILES IN MAIN BRANCH REQUIRES LATEST LLAMA.CPP (May 19th 2023 - commit 2d5db48)!
llama.cpp recently made another breaking change to its quantisation methods - ggerganov/llama.cpp#1508
I have quantised the GGML files in this repo with the latest version. Therefore you will require llama.cpp compiled on May 19th or later (commit 2d5db48 or later) to use them.

For me, it is working with Vigogne-Instruct-13B.ggmlv3.q4_0.bin. But I am on windows, so can't say 100% it will on your machine.

Thanks @Guillaume-Fgt , It works with your workaround !