Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fill_blank报错,换成别的文本进行填空提示Unexpected model output: 26239 #31

Closed
bidoudhd opened this issue Jan 6, 2022 · 3 comments
Labels
question Further information is requested

Comments

@bidoudhd
Copy link

bidoudhd commented Jan 6, 2022

将输入改成
input_text = "近日,北京智源人工智能研究院和清华大学研究团队发布了以中文为核心的大规模预训练语言模型 CPM-LM,参数规模达 26 亿,预训练中文数据规模 100 GB。"
会报错
"Unexpected model output: 26239"
请问fill_blank输入的文本有什么要求?或者对要填空的词有什么要求?
用的是
cpm2 = bminf.models.CPM2()

用pip 安装的,bminf-1.0.0

@bidoudhd bidoudhd closed this as completed Jan 6, 2022
@bidoudhd bidoudhd reopened this Jan 6, 2022
@a710128
Copy link
Collaborator

a710128 commented Jan 7, 2022

在进行填空时,需要通过<span>来设置需要预测的位置。

Input:  近日,北京智源人工智能研究院和清华大学____以中文为核心的大规模预训练语言模型 “____”,参数规模达 26 亿,预训练中文数据规模 100 GB。
Output: 近日,北京智源人工智能研究院和清华大学共同发布以中文为核心的大规模预训练语言模型 “中文语库”,参数规模达 26 亿,预训练中文数据规模 100 GB。

在T5预训练任务,会将15%的token给mask掉,所以在最终使用的时候,合理的设置span会有更好的效果。

@bidoudhd
Copy link
Author

bidoudhd commented Jan 7, 2022

请问对于span的选择有什么要求吗?如果
Input: 近日,北京智源人工智能研究院和清华大学研究团队____以中文为核心的大规模预训练语言模型CPM-LM,参数规模达 26 亿,预训练中文数据规模 100 GB。
则报错
Unexpected model output: 26239
这个错误的原因是什么?

@a710128
Copy link
Collaborator

a710128 commented Jan 7, 2022

这个应该是因为span数量有点少所以模型不能很好的去做填空任务,一般随便再加两个就好了。

@a710128 a710128 added the question Further information is requested label Jan 10, 2022
@a710128 a710128 closed this as completed Jan 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants