We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
我发现在bert4keras读取苏老师开源的chinese_roformer-sim-char-ft_L-12_H-768_A-12和在transformers上读取您的junnyu/roformer_chinese_sim_char_ft_base提取的句向量的效果是差不多的,但是当我尝试将苏老师的模型转为pytorch_bin再用transformers读取后,pooler层怎么都加不上去(bert4keras里指定with_pool='linear'就可以)请问您是如何解决的?
The text was updated successfully, but these errors were encountered:
还有一个问题是在transformers上读取苏老师的模型,它的权重几乎都被重新初始化了导致每次的句向量结果都不一样。。。不知道您是怎么转换的所以transformers可以正常读取?
Sorry, something went wrong.
请使用本仓库的转换代码https://github.com/JunnYu/RoFormer_pytorch/blob/roformer_v2/src/roformer/convert_roformer_original_tf_checkpoint_to_pytorch.py 这个代码对roformer sim用了CasualLM模型加载转换,如果需要pooler的权重请确保RoFormerModel里面的add pooler layer=True。 详细的你可以参考一下,readme里面一个Roformer-sim测试例子。然后有关config,参考https://huggingface.co/junnyu/roformer_chinese_sim_char_ft_small/blob/main/config.json 这里面。
请使用本仓库的转换代码以及roformer模型,如果需要pooler的权重请确保BertModel里面的add pooler layer=True。 详细的你可以参考一下,readme里面一个Roformer-sim测试例子。然后有关config,参考https://huggingface.co/junnyu/roformer_chinese_sim_char_ft_small/blob/main/config.json 这里面。
您好 在用了您仓库的转换代码后已经可以正常读取了 谢谢您!
No branches or pull requests
我发现在bert4keras读取苏老师开源的chinese_roformer-sim-char-ft_L-12_H-768_A-12和在transformers上读取您的junnyu/roformer_chinese_sim_char_ft_base提取的句向量的效果是差不多的,但是当我尝试将苏老师的模型转为pytorch_bin再用transformers读取后,pooler层怎么都加不上去(bert4keras里指定with_pool='linear'就可以)请问您是如何解决的?
The text was updated successfully, but these errors were encountered: