Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

与llama-cpp-python整合出错 #63

Open
Micraow opened this issue Jul 23, 2023 · 1 comment
Open

与llama-cpp-python整合出错 #63

Micraow opened this issue Jul 23, 2023 · 1 comment

Comments

@Micraow
Copy link

Micraow commented Jul 23, 2023

当我尝试用llama-cpp-python 调用chatglm2的q8_0的模型时遇到了问题:

Python 3.11.3 (main, Jun  5 2023, 09:32:32) [GCC 13.1.1 20230429] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from llama_cpp import Llama
>>> llm = Llama(model_path="/home/micraow/chatglm.cpp/chatglm2-ggml.bin")
llama.cpp: loading model from /home/micraow/chatglm.cpp/chatglm2-ggml.bin
error loading model: unknown (magic, version) combination: 6c6d6767, 00000002; is this really a GGML file?
llama_load_model_from_file: failed to load model
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/micraow/aiwork/llama-cpp-python/llama_cpp/llama.py", line 304, in __init__
    assert self.model is not None
           ^^^^^^^^^^^^^^^^^^^^^^
AssertionError

然而用本项目调用则正常,模型应该没有问题,我使用的是llama-cpp-python的最新版本,正在尝试利用privateGPT搭建知识库时遇到了这个问题。不知道该如何解决?

@pengxiulin
Copy link

同样遇到这个问题了,运行环境是 text-generation-webui 和 LM Studio。
对比看了下 llama.cpp 转出来的 ggml 模型文件的Header,两边应该不是同一种封装格式

img_v2_c7655d58-0d51-412c-a37d-5e3937ad52dg

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants