New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

请问作者在预训练中哪一步对数据做了mask？ #36

Closed

ChrisXULC opened this issue Jul 11, 2023 · 1 comment

Labels

question

ChrisXULC commented Jul 11, 2023

请问作者在预训练中哪一步对数据做了mask？

Collaborator

MikeDean2367 commented Jul 11, 2023

您好，感谢关注。我们使用的是transformers包的LlamaForCausalLM类来加载模型，该类提供了相关的代码，您可以阅读相关的源代码。关于具体在哪一步进行mask，您可以参考此处。

zxlzr added the question label

zxlzr closed this as completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment