uSIF vs Averaging

[fros1y]

I noticed that you are calculating sentence embedding using an average of the individual word vectors when performing clustering, etc. Did you happen to evaluate whether SIF or uSIF would be advantageous over averaging?

[yumeng5]

Hi,

Thanks for the question. Are you referring to the following functions in the evaluation code?

Spherical-Text-Embedding/cluster.py

Line 82 in b0f8820

def get_avg_emb(vec_file, text):

Spherical-Text-Embedding/classify.py

Lines 32 to 35 in b0f8820

	def calc_rep(docs, word_emb):
	emb = [np.array([word_emb[w] for w in doc if w in word_emb]) for doc in docs]
	emb = np.array([np.average(vec, axis=0) for vec in emb])
	return emb

For baselines that produce document/sentence embeddings (like SIF and JoSE), we directly take their document/sentence embeddings as features for clustering/classification. The above functions (averaged word embedding) are used to produce sentence embeddings only for word embedding baselines (word2vec) that cannot naturally learn sentence representations. They are actually not used anywhere in the evaluation code (I should have deleted them to avoid confusion).

Please let me know if you have any further questions!

Best,
Yu