Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Chinese datasets #3

Open
xiaohangguo opened this issue Aug 21, 2023 · 3 comments
Open

Chinese datasets #3

xiaohangguo opened this issue Aug 21, 2023 · 3 comments

Comments

@xiaohangguo
Copy link

是这样的，我的数据集是中文+代码的，不知道能不能用您这个微调呢，感觉可以试试看
@juyongjiang
Copy link
Owner

@xiaohangguo 感觉应该是可以的，如果你有数据的话，可以试一试。你如果乐意的话，可以分享你的中文+代码数据在这个仓库下，并对数据进行简单的介绍并pull request，我将非常感激。: )

@xiaohangguo
Copy link
Author

我试了一下，效果不太好，可能是因为我微调的lora，但我把低秩矩阵调到很高了，但效果还是不好，模型没有学习到中文的一些信息，并且知识块也是有问题的

@xiaohangguo
Copy link
Author

xiaohangguo commented Aug 24, 2023 •
edited
Loading

我试了一下，效果不太好，可能是因为我微调的lora，但我把低秩矩阵调到很高了，但效果还是不好，模型没有学习到中文的一些信息，并且知识块也是有问题的

也有可能是，我用的lora训练，训练的参数量太小了

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@juyongjiang @xiaohangguo