About Average

[cxncu001]

2.average vectors, to get representation of the sentence

self.sentence_embeddings = tf.reduce_mean(sentence_embeddings, axis=1) # [None,self.embed_size]

When training, we pad all sentences as max sentence length, then tf.reduce_mean(sentence_embeddings, axis=1) means sum/(max sentence length). How could it know the length of each sentence?

Thanks!

[brightmart]

after pad a sentence to max length of sentence, max length will be the sentence's actual length.

for the word of pad, we will learn a embedding. suppose it is zeros, then it will have no impact to average

[cxncu001]

Thanks.