Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Are uw and us global weights? just to conform. #18

Open
acadTags opened this issue May 4, 2018 · 1 comment
Open

Are uw and us global weights? just to conform. #18

acadTags opened this issue May 4, 2018 · 1 comment

Comments

@acadTags
Copy link

acadTags commented May 4, 2018

Thank you ematvey for this paper.

I wonder the uw and us are two vectors as global weights, or there are different uw(s) for each sentence, and different us(s) for each document?

From the code I think these are global vectors, am I right? Please help me confirm this.

As in the model_components.py it is said

Performs task-specific attention reduction, using learned
attention context vector (constant within task of interest).

The uw or us are defined in the function task_specific_attention(), although they are both referred to the attention_context_vector, but in the computational graph, are they different vectors? It would be helpful if you could explain a little about this part.

attention_context_vector = tf.get_variable(name='attention_context_vector', shape=[output_size], initializer=initializer, dtype=tf.float32)

Thank you.

@dugarsumit
Copy link

I believe you are right. Uw and Us are the global context vectors that stores information about which words or sentences are most informative respectively. They are learned during the training process.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants