Questions about Vector Normalization in TransE

[linzeqipku]

Hi,
I'm interested in your research, but I have some questions about vector normalization in Train_TransE.cpp.

Paper "Translating Embeddings for Modeling Multi-relational Data" says:

The optimization is carried out by stochastic gradient descent (in minibatch mode), over the possible
h, and t, with the additional constraints that the L2-norm of the embeddings of the entities is 1 (no regularization or norm constraints are given to the label embeddings ). This constraint is important for our model, as it is for previous embedding-based methods, because it prevents the training process to trivially minimize L by artificially increasing entity embeddings norms.

But I'm confused about two places in Train_TransE.cpp:

1:
Here is the normalization function in Train_TransE.cpp:
double norm(vector<double> &a) { double x = vec_len(a); if (x>1) for (int ii=0; ii<a.size(); ii++) a[ii]/=x; return 0; }
It seems that you will not normalize vectors whose length is smaller than 1.

2:
Relation vectors are normalized.

Can you give some hints to help me understand your algorithm?

[Mrlyk423]

1. Yes. See our paper "Learning Entity and Relation Embeddings for Knowledge Graph Completion.", the vectors is normalize by |e|<=1 and |r|<=1
2. We also normalize the relation vectors. But indeed, our algorithm have similar performance whether normalize the relation vectors.

Hints: The implement of our code is similar to the pseudocode in the paper "Translating Embeddings for Modeling Multi-relational Data"