Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

代码复现遇到一些问题 #3

Closed
neo-dqy opened this issue Oct 19, 2023 · 2 comments
Closed

代码复现遇到一些问题 #3

neo-dqy opened this issue Oct 19, 2023 · 2 comments

Comments

@neo-dqy
Copy link

neo-dqy commented Oct 19, 2023

您好,我们按readme中的步骤和论文中相同的参数设置,运行了训练和预测的代码。但在预测中,有遇到两个问题:1. 模型对所有三元组都输出True,即使对于训练集的数据也只输出True,虽然训练过程中loss一直在下降;2. 模型预测时有一个warning(A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set padding_side='left' when initializing the tokenizer.)在tokenizer中设置了padding_side='left也没有用。想问问你们有遇到这样的问题吗。

@Zhang-Each
Copy link
Collaborator

关于第一个问题,可以尝试用目前更新的负样本数量更多的数据集版本,也可以自己尝试定义负样本的个数。关于第二个问题,实际推理的时候batch_size为1,因此是否进行padding对结果不影响

@Zhang-Each
Copy link
Collaborator

一般来说,只要经过训练的模型就不可能全出现true的情况,你可以检查一下你的peft库的版本以及checkpoint是否保存下来,peft库存在bug,它的某些版本无法用这套代码保存lora权重,导致模型参数实际上没有更新参数,你可以检查一下你的peft版本以及保存下来的lora权重文件大小是否为0

@Zhang-Each Zhang-Each added help wanted Extra attention is needed and removed help wanted Extra attention is needed labels Dec 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants