-
Notifications
You must be signed in to change notification settings - Fork 43
Home
su edited this page Aug 22, 2018
·
1 revision
Welcome to the RCZoo wiki!
QAnet memory problem: In the HighWay Network, projection layer should be set to decrease the hidden dimension. In the Attention Layer, avoid the similarity tensor's shape expand to [batch_size, q_max_len, p_max_len, hidden_size].
Other details: LayerNorm before each layer, LayerDropout after each layer.