We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
加载模型后,先调用chat(..., stream=True),然后再调chat(..., stream=False)的时候,会报错。
chat(..., stream=True)
chat(..., stream=False)
AttributeError: 'GenerationConfig' object has no attribute 'do_stream'
可能是因为stream里会设置self.__class__.generate 和 sample_stream的原因。
self.__class__.generate
sample_stream
两者行为应该相互独立
from transformers import AutoTokenizer, AutoModelForCausalLM from transformers.generation import GenerationConfig model_id = "Qwen/Qwen-7B-Chat" tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained(model_id, device_map="cuda", trust_remote_code=True).cuda().eval() model.generation_config = GenerationConfig.from_pretrained(model_id, trust_remote_code=True) query = "Hi" history = None model.chat(tokenizer, query, history, stream=True) # <--------- OK model.chat(tokenizer, query, history) # <--------- It will fail ### 运行环境 | Environment ```Markdown - OS: - Python: - Transformers: - PyTorch: - CUDA (`python -c 'import torch; print(torch.version.cuda)'`):
No response
The text was updated successfully, but these errors were encountered:
另外,stop_words_ids在stream模式下是无效的。
stop_words_ids
Sorry, something went wrong.
遇到了同样的问题
建议添加自动测试用例等,提高模型代码质量
@huajianmao 您好,该问题已经在最新的HuggingFace repo中修复,您可以更新到最新版本,然后重新尝试一下 在调用时,请将model.chat(..., stream=True)调用替换为model.chat_stream(...),前者已废弃
model.chat(..., stream=True)
model.chat_stream(...)
更好的做法是不是做下兼容,让 model.chat(..., stream=True)等价于model.chat_stream(...)?
fyabc
No branches or pull requests
是否已有关于该错误的issue? | Is there an existing issue for this?
当前行为 | Current Behavior
加载模型后,先调用
chat(..., stream=True)
,然后再调chat(..., stream=False)
的时候,会报错。可能是因为stream里会设置
self.__class__.generate
和sample_stream
的原因。期望行为 | Expected Behavior
两者行为应该相互独立
复现方法 | Steps To Reproduce
备注 | Anything else?
No response
The text was updated successfully, but these errors were encountered: