Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

按照README上面的实例运行报错 #15

Closed
sxk000 opened this issue Feb 28, 2023 · 2 comments
Closed

按照README上面的实例运行报错 #15

sxk000 opened this issue Feb 28, 2023 · 2 comments

Comments

@sxk000
Copy link

sxk000 commented Feb 28, 2023

感谢大佬分享!

我在按照README上面的实例:

加载模型

from transformers import T5Tokenizer, T5ForConditionalGeneration
tokenizer = T5Tokenizer.from_pretrained("ClueAI/ChatYuan-large-v1")
model = T5ForConditionalGeneration.from_pretrained("ClueAI/ChatYuan-large-v1")

使用

import torch
from transformers import AutoTokenizer

修改colab笔记本设置为gpu,推理更快

device = torch.device('cuda')
model.to(device)
def preprocess(text):
text = text.replace("\n", "\n").replace("\t", "\t")
return text

def postprocess(text):
return text.replace("\n", "\n").replace("\t", "\t")

def answer(text, sample=True, top_p=1, temperature=0.7):
'''sample:是否抽样。生成任务,可以设置为True;
top_p:0-1之间,生成的内容越多样'''
text = preprocess(text)
encoding = tokenizer(text=[text], truncation=True, padding=True, max_length=768, return_tensors="pt").to(device)
if not sample:
out = model.generate(**encoding, return_dict_in_generate=True, output_scores=False, max_new_tokens=512, num_beams=1, length_penalty=0.6)
else:
out = model.generate(**encoding, return_dict_in_generate=True, output_scores=False, max_new_tokens=512, do_sample=True, top_p=top_p, temperature=temperature, no_repeat_ngram_size=3)
out_text = tokenizer.batch_decode(out["sequences"], skip_special_tokens=True)
return postprocess(out_text[0])
print("end...")

input_text0 = "帮我写一个请假条,我因为新冠不舒服,需要请假3天,请领导批准"
input_text1 = "你能干什么"
input_text2 = "写一封英文商务邮件给英国客户,表达因为物流延误,不能如期到达,我们可以赔偿贵公司所有损失"
input_text3 = "写一个文章,题目是未来城市"
input_text4 = "写一个诗歌,关于冬天"
input_text5 = "从南京到上海的路线"
input_text6 = "学前教育专业岗位实习中,在学生方面会存在问题,请提出改进措施。800字"
input_text7 = "根据标题生成文章:标题:屈臣氏里的化妆品到底怎么样?正文:化妆品,要讲究科学运用,合理搭配。屈臣氏起码是正品连锁店。请继续后面的文字。"
input_text8 = "帮我对比几款GPU,列出详细参数对比,并且给出最终结论"
input_list = [input_text0, input_text1, input_text2, input_text3, input_text4, input_text5, input_text6, input_text7, input_text8]
for i, input_text in enumerate(input_list):
input_text = "用户:" + input_text + "\n小元:"
print(f"示例{i}".center(50, "="))
output_text = answer(input_text)
print(f"{input_text}{output_text}")

报如下错误:

=======================示例0========================
Traceback (most recent call last):
File "", line 4, in
File "", line 9, in answer
File "/home/anaconda3/envs/s20230220e310fsb/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/home/anaconda3/envs/s20230220e310fsb/lib/python3.10/site-packages/transformers/generation/utils.py", line 1252, in generate
model_kwargs = self._prepare_encoder_decoder_kwargs_for_generation(
File "/home/anaconda3/envs/s20230220e310fsb/lib/python3.10/site-packages/transformers/generation/utils.py", line 617, in _prepare_encoder_decoder_kwargs_for_generation
model_kwargs["encoder_outputs"]: ModelOutput = encoder(**encoder_kwargs)
File "/home/anaconda3/envs/s20230220e310fsb/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/home/anaconda3/envs/s20230220e310fsb/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1055, in forward
layer_outputs = layer_module(
File "/home/anaconda3/envs/s20230220e310fsb/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/home/anaconda3/envs/s20230220e310fsb/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward
self_attention_outputs = self.layer[0](
File "/home/anaconda3/envs/s20230220e310fsb/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/home/anaconda3/envs/s20230220e310fsb/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 593, in forward
attention_output = self.SelfAttention(
File "/home/anaconda3/envs/s20230220e310fsb/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/home/anaconda3/envs/s20230220e310fsb/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 512, in forward
query_states = shape(self.q(hidden_states)) # (batch_size, n_heads, seq_length, dim_per_head)
File "/home/anaconda3/envs/s20230220e310fsb/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/home/anaconda3/envs/s20230220e310fsb/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 114, in forward
return F.linear(input, self.weight, self.bias)
RuntimeError: CUDA error: CUBLAS_STATUS_INVALID_VALUE when calling cublasSgemm( handle, opa, opb, m, n, k, &alpha, a, lda, b, ldb, &beta, c, ldc)

@joytianya
Copy link
Collaborator

看着像是环境配置问题,可以先用colab跑下试试

@sxk000
Copy link
Author

sxk000 commented Mar 2, 2023

看着像是环境配置问题,可以先用colab跑下试试

是的,重新安装了pytorch,可以了,谢谢你

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants