Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DIEN组网代码一点疑惑 #713

Closed
tz28 opened this issue Feb 23, 2022 · 3 comments
Closed

DIEN组网代码一点疑惑 #713

tz28 opened this issue Feb 23, 2022 · 3 comments

Comments

@tz28
Copy link

tz28 commented Feb 23, 2022

DIEN组网代码中,下面这两行调用的add_sublayer()函数【self.add_sublayer('linear_%d' % i, linear)】,在传入的name都是'linear_%d' % i的情况下,是否会后面即L144里的覆盖L123里的,如果会覆盖,这么操作两遍add_sublayer()的目的是什么?
https://github.com/PaddlePaddle/PaddleRec/blob/master/models/rank/dien/net.py#L123
https://github.com/PaddlePaddle/PaddleRec/blob/master/models/rank/dien/net.py#L144

@wangzhen38
Copy link
Collaborator

两个名字不该重复的,我们会尽快修改,感谢反馈~

@tz28
Copy link
Author

tz28 commented Feb 23, 2022

两个名字不该重复的,我们会尽快修改,感谢反馈~

好的,点赞

@tz28
Copy link
Author

tz28 commented Feb 24, 2022

两个名字不该重复的,我们会尽快修改,感谢反馈~

另外,还有两个问题,

  1. 在实现dien论文中的辅助loss即aux_loss(论文中的公式6)时,漏了下面公式中的 1-
    image

代码链接:https://github.com/PaddlePaddle/PaddleRec/blob/master/models/rank/dien/net.py#L243

  1. DIEN论文中在Interest Evolving Layer(下图中粉红色部分)采用的是改造后的GRU,即AUGRU,其公式如下公式所示,而paddle实现的版本中直接用了最普通的GRU,然后把attention权重部分直接与输入做了点乘后输入到标注GRU中,显然与论文不符合,我看了论文作者开源的代码中自己实现了AUGRU,参见:https://github.com/mouna99/dien/blob/master/script/utils.py#L142 因此,paddle这种实现得到的结果,是不置信的。从paddle的宣传来看,这些模型是其宣传支持的重点,且不说要100%和原论文符合,起码论文的创新点实现要符合,否则就是个demo,与paddle的工业实践宣传不符合。

image

image

@tz28 tz28 closed this as completed Mar 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants