运行 python init_database --recreate-vs 报错是什么原因？ #2216
Comments
|
确定模型文件完整么 |
|
模型文件完整的，在终端运行会有这个报错，后来在pycharm中运行可以成功，但是运行startup.py后可以正常显示，但调用对话后报错：APIConnectionError:Caught exception: Error communicating with OpenAI |
|
Error communicating with OpenAI 其实就是没有连接上你本地的大模型，那个报错通常是Embed模型不完整，其他错误我们暂时复现不出来 |
|
碰到了同样的问题，试试这个 (p11) PS D:\code\Langchain-Chatchat-master> python .\init_database.py --recreate-vs
recreating all vector stores
2023-12-01 19:56:20,532 - faiss_cache.py[line:80] - INFO: loading vector store in 'samples/vector_store/m3e-base' from disk.
['init'] m3e-base False
2023-12-01 19:56:21,079 - SentenceTransformer.py[line:66] - INFO: Load pretrained SentenceTransformer: moka-ai/m3e-base
.gitattributes: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 1.53k/1.53k [00:00<?, ?B/s]
1_Pooling/config.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 190/190 [00:00<?, ?B/s]
README.md: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 26.6k/26.6k [00:00<00:00, 671kB/s]
config.json: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████| 932/932 [00:00<?, ?B/s]
model.safetensors: 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 409M/409M [00:16<00:00, 24.2MB/s]
pytorch_model.bin: 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 409M/409M [00:14<00:00, 28.0MB/s]
sentence_bert_config.json: 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 53.0/53.0 [00:00<?, ?B/s]
special_tokens_map.json: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 125/125 [00:00<?, ?B/s]
tokenizer.json: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 439k/439k [00:00<00:00, 758kB/s]
tokenizer_config.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 342/342 [00:00<?, ?B/s]
vocab.txt: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 110k/110k [00:00<00:00, 5.51MB/s]
modules.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████| 229/229 [00:00<?, ?B/s]
Batches: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 21.96it/s]
2023-12-01 19:57:06,293 - loader.py[line:54] - INFO: Loading faiss with AVX2 support.
2023-12-01 19:57:06,293 - loader.py[line:58] - INFO: Could not load library with AVX2 support due to:
ModuleNotFoundError("No module named 'faiss.swigfaiss_avx2'") |
|
为啥我是这样 recreating all vector stores The above exception was the direct cause of the following exception: File "E:\Prj\ChatGLM3-6B-32K\venv\lib\site-packages\langchain_core\vectorstores.py", line 510, in from_documents |
你这是模型下载失败了 |
我下载了glm3的模型了 |
2023-12-02 01:05:52,895 - faiss_cache.py[line:80] - INFO: loading vector store in 'samples/vector_store/m3e-base' from disk. embedding 模型下载失败了 |
拉取项目配置好环境后：
运行 python init_database --recreate-vs 报错
报错内容：
recreating all vector stores
2023-11-29 15:52:26,898 - faiss_cache.py[line:80] - INFO: loading vector store in 'samples/vector_store/m3e-base' from disk.
2023-11-29 15:52:26,904 - embeddings_api.py[line:39] - ERROR: Could not import sentence_transformers python package. Please install it with
pip install sentence-transformers.AttributeError: 'NoneType' object has no attribute 'conjugate'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/ps/lpl/Langchain-Chatchat/init_database.py", line 108, in
folder2db(kb_names=args.kb_name, mode="recreate_vs", embed_model=args.embed_model)
File "/home/ps/lpl/Langchain-Chatchat/server/knowledge_base/migrate.py", line 118, in folder2db
kb.create_kb()
File "/home/ps/lpl/Langchain-Chatchat/server/knowledge_base/kb_service/base.py", line 80, in create_kb
self.do_create_kb()
File "/home/ps/lpl/Langchain-Chatchat/server/knowledge_base/kb_service/faiss_kb_service.py", line 47, in do_create_kb
self.load_vector_store()
File "/home/ps/lpl/Langchain-Chatchat/server/knowledge_base/kb_service/faiss_kb_service.py", line 28, in load_vector_store
return kb_faiss_pool.load_vector_store(kb_name=self.kb_name,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ps/lpl/Langchain-Chatchat/server/knowledge_base/kb_cache/faiss_cache.py", line 90, in load_vector_store
vector_store = self.new_vector_store(embed_model=embed_model, embed_device=embed_device)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ps/lpl/Langchain-Chatchat/server/knowledge_base/kb_cache/faiss_cache.py", line 48, in new_vector_store
vector_store = FAISS.from_documents([doc], embeddings, normalize_L2=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ps/anaconda3/envs/langchain_sengoic_env/lib/python3.11/site-packages/langchain/schema/vectorstore.py", line 510, in from_documents
return cls.from_texts(texts, embedding, metadatas=metadatas, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ps/anaconda3/envs/langchain_sengoic_env/lib/python3.11/site-packages/langchain/vectorstores/faiss.py", line 911, in from_texts
embeddings = embedding.embed_documents(texts)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ps/lpl/Langchain-Chatchat/server/knowledge_base/kb_service/base.py", line 380, in embed_documents
return normalize(embeddings).tolist()
^^^^^^^^^^^^^^^^^^^^^
File "/home/ps/lpl/Langchain-Chatchat/server/knowledge_base/kb_service/base.py", line 37, in normalize
norm = np.linalg.norm(embeddings, axis=1)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ps/anaconda3/envs/langchain_sengoic_env/lib/python3.11/site-packages/numpy/linalg/linalg.py", line 2582, in norm
s = (x.conj() * x).real
^^^^^^^^
TypeError: loop of ufunc does not support argument 0 of type NoneType which has no callable conjugate method
运行环境：
python版本:3.11.5
site_pacjages:
accelerate 0.24.1
aiohttp 3.9.1
aiolimiter 1.1.0
aiosignal 1.3.1
altair 5.1.2
antlr4-python3-runtime 4.9.3
anyio 3.7.1
attrs 23.1.0
backoff 2.2.1
bce-python-sdk 0.8.97
beautifulsoup4 4.12.2
blinker 1.7.0
blis 0.7.11
Brotli 1.1.0
cachetools 5.3.2
catalogue 2.0.10
certifi 2023.11.17
cffi 1.16.0
chardet 5.2.0
charset-normalizer 3.3.2
click 8.1.7
cloudpathlib 0.16.0
coloredlogs 15.0.1
confection 0.1.4
contourpy 1.2.0
cryptography 41.0.5
cycler 0.12.1
cymem 2.0.8
dashscope 1.13.3
dataclasses 0.6
dataclasses-json 0.6.2
distro 1.8.0
effdet 0.4.1
einops 0.7.0
emoji 2.8.0
et-xmlfile 1.1.0
faiss-cpu 1.7.4
fastapi 0.104.1
filelock 3.13.1
filetype 1.2.0
flatbuffers 23.5.26
fonttools 4.45.1
frozenlist 1.4.0
fschat 0.2.33
fsspec 2023.10.0
future 0.18.3
gitdb 4.0.11
GitPython 3.1.40
greenlet 3.0.1
h11 0.14.0
h2 4.1.0
hpack 4.0.0
httpcore 1.0.2
httptools 0.6.1
httpx 0.25.2
huggingface-hub 0.19.4
humanfriendly 10.0
hyperframe 6.0.1
idna 3.6
importlib-metadata 6.8.0
iniconfig 2.0.0
iopath 0.1.10
Jinja2 3.1.2
joblib 1.3.2
jsonpatch 1.33
jsonpointer 2.4
jsonschema 4.20.0
jsonschema-specifications 2023.11.1
kiwisolver 1.4.5
langchain 0.0.340
langchain-experimental 0.0.42
langcodes 3.3.0
langdetect 1.0.9
langsmith 0.0.66
layoutparser 0.3.4
lxml 4.9.3
Markdown 3.5.1
markdown-it-py 3.0.0
markdown2 2.4.10
markdownify 0.11.6
MarkupSafe 2.1.3
marshmallow 3.20.1
matplotlib 3.8.2
mdurl 0.1.2
mpmath 1.3.0
msg-parser 1.2.0
msgpack 1.0.7
multidict 6.0.4
murmurhash 1.0.10
mypy-extensions 1.0.0
networkx 3.2.1
nh3 0.2.14
ninja 1.11.1.1
nltk 3.8.1
numexpr 2.8.7
numpy 1.26.2
nvidia-cublas-cu12 12.1.3.1
nvidia-cuda-cupti-cu12 12.1.105
nvidia-cuda-nvrtc-cu12 12.1.105
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu12 8.9.2.26
nvidia-cufft-cu12 11.0.2.54
nvidia-curand-cu12 10.3.2.106
nvidia-cusolver-cu12 11.4.5.107
nvidia-cusparse-cu12 12.1.0.106
nvidia-nccl-cu12 2.18.1
nvidia-nvjitlink-cu12 12.3.101
nvidia-nvtx-cu12 12.1.105
olefile 0.46
omegaconf 2.3.0
onnx 1.15.0
onnxruntime 1.15.1
openai 0.28.1
opencv-python 4.8.1.78
openpyxl 3.1.2
packaging 23.2
pandas 2.1.3
pathlib 1.0.1
pdf2image 1.16.3
pdfminer.six 20221105
pdfplumber 0.10.3
peft 0.6.2
Pillow 9.5.0
pip 23.3.1
pluggy 1.3.0
portalocker 2.8.2
preshed 3.0.9
prompt-toolkit 3.0.41
protobuf 4.25.1
psutil 5.9.6
pyarrow 14.0.1
pyclipper 1.3.0.post5
pycocotools 2.0.7
pycparser 2.21
pycryptodome 3.19.0
pydantic 1.10.13
pydeck 0.8.1b0
Pygments 2.17.2
PyJWT 2.8.0
PyMuPDF 1.23.6
PyMuPDFb 1.23.6
pypandoc 1.12
pyparsing 3.1.1
pypdfium2 4.24.0
pytesseract 0.3.10
pytest 7.4.3
python-dateutil 2.8.2
python-decouple 3.8
python-docx 1.1.0
python-dotenv 1.0.0
python-iso639 2023.6.15
python-magic 0.4.27
python-multipart 0.0.6
python-pptx 0.6.23
pytz 2023.3.post1
PyYAML 6.0.1
qianfan 0.1.1
rapidfuzz 3.5.2
rapidocr-onnxruntime 1.3.8
ray 2.8.0
referencing 0.31.0
regex 2023.10.3
requests 2.31.0
rich 13.7.0
rpds-py 0.13.1
safetensors 0.4.0
scikit-learn 1.3.2
scipy 1.11.4
sentence-transformers 2.2.2
sentencepiece 0.1.99
setuptools 68.0.0
shapely 2.0.2
shortuuid 1.0.11
simplejson 3.19.2
six 1.16.0
smart-open 6.4.0
smmap 5.0.1
sniffio 1.3.0
socksio 1.0.0
soupsieve 2.5
spacy 3.7.2
spacy-legacy 3.0.12
spacy-loggers 1.0.5
SQLAlchemy 2.0.23
srsly 2.4.8
starlette 0.27.0
streamlit 1.28.2
streamlit-aggrid 0.3.4.post3
streamlit-antd-components 0.2.3
streamlit-chatbox 1.1.11
streamlit-feedback 0.1.3
streamlit-option-menu 0.3.6
strsimpy 0.2.1
svgwrite 1.4.3
sympy 1.12
tabulate 0.9.0
tenacity 8.2.3
thinc 8.2.1
threadpoolctl 3.2.0
tiktoken 0.5.1
timm 0.9.12
tokenizers 0.15.0
toml 0.10.2
toolz 0.12.0
torch 2.1.0
torchaudio 2.1.0
torchvision 0.16.0
tornado 6.3.3
tqdm 4.66.1
transformers 4.35.2
transformers-stream-generator 0.0.4
triton 2.1.0
typer 0.9.0
typing_extensions 4.8.0
typing-inspect 0.9.0
tzdata 2023.3
tzlocal 5.2
unstructured 0.11.0
unstructured-inference 0.7.15
unstructured.pytesseract 0.3.12
urllib3 2.1.0
uvicorn 0.24.0.post1
uvloop 0.19.0
validators 0.22.0
vllm 0.2.2
wasabi 1.1.2
watchdog 3.0.0
watchfiles 0.21.0
wavedrom 2.0.3.post3
wcwidth 0.2.12
weasel 0.3.4
websockets 12.0
wheel 0.41.2
wrapt 1.16.0
xformers 0.0.22.post7
xlrd 2.0.1
XlsxWriter 3.1.9
yarl 1.9.3
zhipuai 1.0.7
zipp 3.17.0
The text was updated successfully, but these errors were encountered: