This repository was archived by the owner on Jul 7, 2023. It is now read-only.

Description
The common_attention method dot_product_attention_relative requires that the query and value have compatible depth. See
|
q.get_shape().assert_is_compatible_with(v.get_shape()) |
While that's certainly true for the query and key, the value is handled separately and doesn't interact. I've commented out the compatibility assertion and verified that the method runs well and models using it train well. Am I missing something here or can this line be removed?