You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Nov 22, 2022. It is now read-only.
This is correct. The current implementation doesn't account for multi-hop attention. In effect it doesn't implement equation (6) in the paper but stops at equation (5). Thanks for pointing it out. We will get back on the fix which requires introducing a new parameter r which controls the number of attention hops.
in the original paper, there are two equations for attention, (5) and (6) (on page 3).
near as we can tell, in the code - https://github.com/facebookresearch/pytext/blob/master/pytext/models/representations/pooling.py#L22 - the ws2 tensor is of the wrong dimension; i.e. it's only of dim, in language of the paper, d_a instead of a matrix of r x d_a.
is this intentional?
thanks!
The text was updated successfully, but these errors were encountered: