chatglm3-6b模型llm模式，正常对话突然报错TypeError: object of type 'NoneType' has no len()

[Andy1018]

24-03-13 17:02:32,422 - utils.py[line:38] - ERROR: object of type 'NoneType' has no len()
Traceback (most recent call last):
File "/home/chatglm3/Langchain-Chatchat/server/utils.py", line 36, in wrap_done
await fn
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain/chains/base.py", line 385, in acall
raise e
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain/chains/base.py", line 379, in acall
await self._acall(inputs, run_manager=run_manager)
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain/chains/llm.py", line 275, in _acall
response = await self.agenerate([inputs], run_manager=run_manager)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain/chains/llm.py", line 142, in agenerate
return await self.llm.agenerate_prompt(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 554, in agenerate_prompt
return await self.agenerate(
^^^^^^^^^^^^^^^^^^^^^
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 514, in agenerate
raise exceptions[0]
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 617, in _agenerate_with_cache
return await self._agenerate(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain_community/chat_models/openai.py", line 522, in _agenerate
return await agenerate_from_stream(stream_iter)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 87, in agenerate_from_stream
async for chunk in stream:
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain_community/chat_models/openai.py", line 493, in _astream
if len(chunk["choices"]) == 0:
^^^^^^^^^^^^^^^^^^^^^
TypeError: object of type 'NoneType' has no len()
2024-03-13 17:02:32,426 - utils.py[line:40] - ERROR: TypeError: Caught exception: object of type 'NoneType' has no len()
2024-03-13 17:02:34,294 - utils.py[line:38] - ERROR: object of type 'NoneType' has no len()
Traceback (most recent call last):
File "/home/chatglm3/Langchain-Chatchat/server/utils.py", line 36, in wrap_done
await fn
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain/chains/base.py", line 385, in acall
raise e
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain/chains/base.py", line 379, in acall
await self._acall(inputs, run_manager=run_manager)
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain/chains/llm.py", line 275, in _acall
response = await self.agenerate([inputs], run_manager=run_manager)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain/chains/llm.py", line 142, in agenerate
return await self.llm.agenerate_prompt(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 554, in agenerate_prompt
return await self.agenerate(
^^^^^^^^^^^^^^^^^^^^^
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 514, in agenerate
raise exceptions[0]
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 617, in _agenerate_with_cache
return await self._agenerate(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain_community/chat_models/openai.py", line 522, in _agenerate
return await agenerate_from_stream(stream_iter)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 87, in agenerate_from_stream
async for chunk in stream:
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain_community/chat_models/openai.py", line 493, in _astream
if len(chunk["choices"]) == 0:
^^^^^^^^^^^^^^^^^^^^^
TypeError: object of type 'NoneType' has no len()
2024-03-13 17:02:34,296 - utils.py[line:40] - ERROR: TypeError: Caught exception: object of type 'NoneType' has no len()
2024-03-13 17:02:41,976 - utils.py[line:38] - ERROR: object of type 'NoneType' has no len()
Traceback (most recent call last):
File "/home/chatglm3/Langchain-Chatchat/server/utils.py", line 36, in wrap_done
await fn
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain/chains/base.py", line 385, in acall
raise e
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain/chains/base.py", line 379, in acall
await self._acall(inputs, run_manager=run_manager)
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain/chains/llm.py", line 275, in _acall
response = await self.agenerate([inputs], run_manager=run_manager)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain/chains/llm.py", line 142, in agenerate
return await self.llm.agenerate_prompt(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 554, in agenerate_prompt
return await self.agenerate(
^^^^^^^^^^^^^^^^^^^^^
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 514, in agenerate
raise exceptions[0]
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 617, in _agenerate_with_cache
return await self._agenerate(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain_community/chat_models/openai.py", line 522, in _agenerate
return await agenerate_from_stream(stream_iter)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 87, in agenerate_from_stream
async for chunk in stream:
File "/home/user/anaconda3/envs/langchai-chat/lib/python3.11/site-packages/langchain_community/chat_models/openai.py", line 493, in _astream
if len(chunk["choices"]) == 0:
^^^^^^^^^^^^^^^^^^^^^
TypeError: object of type 'NoneType' has no len()
2024-03-13 17:02:41,978 - utils.py[line:40] - ERROR: TypeError: Caught exception: object of type 'NoneType' has no len()

[dosubot]

嗨，@Andy1018！很高兴再次见到你。遇到了新的挑战吗？让我们一起看看这次的问题。

您遇到的 "TypeError: object of type 'NoneType' has no len()" 错误可能是由于在期望具有定义长度的对象（如列表、字符串或任何其他集合类型）的操作中接收到了 None 值。这种错误通常在尝试对 None 值使用 len() 等函数时发生。

在您提供的代码片段中，可能出现此错误的一行是：

if len(chunk["choices"]) == 0:

这行代码的设计是为了检查 chunk["choices"] 的长度。然而，如果 chunk 或 chunk["choices"] 是 None，调用 len(chunk["choices"]) 将引发 "TypeError: object of type 'NoneType' has no len()" 错误，因为 len() 不能用于 None 值。

为了解决这个问题，您需要确保 chunk 和 chunk["choices"] 在执行长度检查之前不是 None。您可以在执行长度检查之前添加一个检查，例如：

if chunk is not None and chunk["choices"] is not None:
    if len(chunk["choices"]) == 0:
        continue

这个检查将防止在 chunk 或 chunk["choices"] 为 None 时执行长度检查，从而避免 "TypeError: object of type 'NoneType' has no len()" 错误。

此外，我在 Langchain-Chatchat 仓库中找到了一些类似的问题，可能会对您有所帮助。这些问题的解决方案包括确保 GPU 内存不满、在没有 GPU 的服务器上运行模型时考虑切换到支持 CPU 的模型、升级 macOS 版本以解决与 int64 数据类型支持相关的问题等。具体详情请参考上下文中的问题和答案部分。

希望这些信息对您有所帮助。如果您有任何其他问题或需要进一步的帮助，请随时告诉我。

Sources

• [BUG] mac m1启动后，本地chatGLM6b模型问答报错 utils.py[line:24] - ERROR: object of type 'NoneType' has no len()
• TypeError: object of type 'NoneType' has no len()
• 执行python startup.py -a 报错
• 历史对话过程报错: utils.py[line:25] - ERROR: TypeError: Caught exception: object of type 'NoneType' has no len()
• server/agent/custom_agent/ChatGLM3Agent.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[zRzRzRzRzRzRzR]

估计是说一半模型异常了，你单独跑chatglm3的示例能正常回答完整的流程吗

[Donovan-Ye]

是不是显存不够了。。我们这边爆显存的时候会出现这个错误

[LaoK263]

我使用智谱的在线API，本地没有大模型，也遇到了相同的错误，每次出现该错误时，chat都没有返回，应该是对超时未返回或者返回异常的处理问题，一般情况下重试就行了。

[sevenandseven]

估计是说一半模型异常了，你单独跑chatglm3的示例能正常回答完整的流程吗

你好，我在hf上下载了128k的长文本模型，但是他在langchain工程中使用时，出现了utils.py[line:27] - ERROR: TypeError: Caught exception: object of type 'NoneType' has no len() 问题，并且该模型无法进行正常对话，请问怎么处理？

[xuzhigong]

windows无独立显卡跑本地模型（千问）会遇到这个问题，切换为在线模型后可正常使用

[Buezw]

torch 版本不对

[Andy1018]

@Buezw torch版本不对还能正常使用？是在正常对话的过程中出现了问题

[Andy1018]

估计是说一半模型异常了，你单独跑chatglm3的示例能正常回答完整的流程吗

你好，我在hf上下载了128k的长文本模型，但是他在langchain工程中使用时，出现了utils.py[line:27] - ERROR: TypeError: Caught exception: object of type 'NoneType' has no len() 问题，并且该模型无法进行正常对话，请问怎么处理？

我觉得这个是正解