[BUG] ERROR: TypeError: Caught exception: object of type 'NoneType' has no len()

[lhtpluto]

问题描述 / Problem Description
用简洁明了的语言描述这个问题 / Describe the problem in a clear and concise manner.

使用Qwen1.5-72B-Chat时，chat回答一半报错ERROR: TypeError: Caught exception: object of type 'NoneType' has no len()
使用Qwen1.5-72B-Chat-GPTQ-Int8时，chat回答直接报错ERROR: TypeError: Caught exception: object of type 'NoneType' has no len()

操作系统：Linux-5.15.146.1-microsoft-standard-WSL2-x86_64-with-glibc2.35.
python版本：3.11.7 (main, Dec 15 2023, 18:12:31) [GCC 11.2.0]
项目版本：v0.2.10
langchain版本：0.0.354. fastchat版本：0.2.35

当前使用的分词器：ChineseRecursiveTextSplitter
当前启动的LLM模型：['Qwen1.5-72B-Chat'] @ cuda
{'device': 'cuda',
'host': '0.0.0.0',
'infer_turbo': False,
'model_path': 'Qwen1.5-72B-Chat',
'model_path_exists': True,
'port': 20002}
当前Embbedings模型： bge-large-zh-v1.5 @ cuda

环境信息 / Environment Information

• langchain-ChatGLM 版本/commit 号：v0.2.10
• 是否使用 Docker 部署（是/否）：否
• 使用的模型（ChatGLM2-6B / Qwen-7B 等）：Qwen1.5-72B-Chat
• 使用的 Embedding 模型（moka-ai/m3e-base 等）：bge-large-zh-v1.5
• 使用的向量库类型 (faiss / milvus / pg_vector 等)： faiss
• 操作系统及版本 / Operating system and version: WSL
• Python 版本 / Python version: 3.11.7
• 其他相关环境信息 / Other relevant environment information:

附加信息 / Additional Information
添加与问题相关的任何其他信息 / Add any other information related to the issue.

[dosubot]

The TypeError: Caught exception: object of type 'NoneType' has no len() error you're encountering seems to be due to a function expecting a collection or sequence but instead receiving a None value. This issue indicates a bug or an unexpected state during runtime. Here are some targeted solutions to address this problem:

1. Ensure Iterable Objects: Make sure running_models is always an iterable by defaulting to an empty list if api.list_running_models() returns None. This can be done with:

   running_models = list(api.list_running_models() or [])

2. Validate api.list_running_models() Return Value: Confirm that the api.list_running_models() function returns a list of models instead of None. Debug the endpoint and request to ensure they're functioning correctly. If the endpoint and request work as expected but return None, add a check before iterating to ensure that models is not None:

   models = r.json().get("models")
   if models is None:
       logger.error('No models returned from controller')
       return BaseResponse(code=500, data={}, msg='No models returned from controller')

3. Check Before Length Operation: Before attempting to get the length of chunk["choices"], check if it is None to prevent the TypeError. This can be implemented as:

   if chunk["choices"] is not None:
       if len(chunk["choices"]) == 0:
           # Your code here

   Or, ensure chunk["choices"] defaults to an empty list:

   chunk["choices"] = chunk.get("choices", [])
   if len(chunk["choices"]) == 0:
       # Your code here

These solutions are derived from similar issues found in the Langchain-Chatchat repository. They aim to address the problem by ensuring that the code does not attempt to operate on None as if it were a collection or sequence. If you need further assistance or have more questions, feel free to ask.

Sources

• TypeError: 'NoneType' object is not iterable
• [BUG] 简洁阐述问题 / Concise description of the issueTraceback (most recent call last): File "/data/soft/anaconda3/envs/langchain/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 535, in _run_script exec(code, module.dict) File "/data/chenhui/text_similarity_main/Langchain-Chatchat/webui.py", line 64, in pages[selected_page]["func"](api=api, is_lite=is_lite) File "/data/chenhui/text_similarity_main/Langchain-Chatchat/webui_pages/dialogue/dialogue.py", line 163, in dialogue_page running_models = list(api.list_running_models()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: 'NoneType' object is not iterable 报错的原因，给我正确的答案
• 对话时输入问题报错ERROR: object of type 'NoneType' has no len()
• ['Qwen-1_8B-Chat'] cpu]('Qwen-1_8B-Chat'] cpu #3497)
• 对数据库 提问有问题，找到数据但不回答或数据库没有的什么都不回答
• 对数据库提问报错
• server/model_workers/qwen.py
• server/llm_api_stale.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[zRzRzRzRzRzRzR]

模型没有正常返回的错误都是这样的

[xinshang091720]

这个错误怎么修改啊