Add KEN to leverage numerical attributes in KG embeddings

[alexis-cvetkov]

Publication Link

https://hal.science/hal-03848124/document

Reference Implementation

https://github.com/alexis-cvetkov/KEN

Additional Implementations

No response

Relevance

KEN (Knowledge Embedding with Numbers) extends KG embedding models to numerical triples, e.g. (Paris, HasPopulation, 2.2M). The idea is the following: a numerical value $x$ from an attribute/relation $r$ is embedded into the same space as other entities: $e_r(x) = ReLU(x * w_r + b_r) \in \mathbb{R}^p$.

Before embedding them, we normalize the values $x$ between 0 and 1, using a quantile transform.

We compared this model to LiteralE, and it performed better. Although we evaluated it on downstream tasks (external to the KG), and not on link prediction. We also did an ablation study to show the importance of the quantile transform and the ReLU activation.

Additional Context

I am the author of the model and have implemented the approach on top of PyKEEN. Now I would like to add it to the core package.

[mberr]

Hi @alexis-cvetkov ,

we would appreciate your PR, especially as literal KG models have been a niche topic within PyKEEN. I would suggest you go ahead an open a PR with your suggested addition and we can give you feedback over there.

I did not yet have the time to fully read your paper, but from skimming over it it seems like you are also evaluating on the prediction of numerical attributes, which is something we do not yet have within PyKEEN. Maybe we can leave this part for a follow-up PR? Happy to hear your thoughts on this though.

EDIT: It sounds like the quantile transform is a somewhat orthogonal component to the specific prediction, $e_r(x) = ReLU(x * w_r + b_r) \in \mathbb{R}^p$. I assume we could implement this as some generic pre-/postprocessing of the numeric attributes?