Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

论文中的网络结构与开源代码网络结构不同 #8

Closed
FunkyKoki opened this issue Aug 17, 2022 · 1 comment
Closed

论文中的网络结构与开源代码网络结构不同 #8

FunkyKoki opened this issue Aug 17, 2022 · 1 comment

Comments

@FunkyKoki
Copy link

FunkyKoki commented Aug 17, 2022 •
edited

感谢作者！但是还有一个疑问想请教一下。
在论文中，特意列出了Table 5，以论证learned tokenizer是弱于atten-based tokenizer的，但是在代码中有query项，这个query就是自行学习的，这也使得最后的attention map也是基于query对原始特征的变换后得到的。这里面是否有论文与代码不一致的问题呢？
@FunkyKoki FunkyKoki reopened this Aug 17, 2022
@FunkyKoki
Copy link
Author

FunkyKoki commented Aug 17, 2022 •
edited

我大概理解了，query就是图片中conv2d的参数，只是变成了对序列做处理的形式

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@FunkyKoki