💃 🥤 Extract interaction function from models

[mberr]

This is a replacement for #88 , where the merge target is master.

@mali-git As discussed in today's call, I tried to draft an API for the interaction function. It is built around the most generic form of interaction function, which has one batch dimension, and then allows broadcasting over multiple entities/relations to meet the use cases for e.g. scoring all tail entities at once, but also supports, e.g. full CWA scores.

This can likely also help for the fast LCWA @lvermue once envisioned 😉

I did not define the methods to be static to allow for parametric interaction functions such as e.g. ER-MLP having some weights.

Overview

• One shared implementation for score_hrt / score_h / score_r / score_t in the base class, done in InteractionFunction.
• One shared implementation of _score for all models sharing the same set of embeddings (e.g. TransE/DistMult/ERMLP -> one vector for each entity/relation, TransH -> additional vector for each entity, etc.)
• A state-less functional form of the interaction function where all necessary states are passed from the outside. This is done in pykeen.nn.modules.
• A state-full implementation of interaction function encapsulating all shared parameters (e.g. weight matrices for ERMLP / ConvE, etc.), but delegating the actual interaction to the state-less version. This is done in pykeen.nn.functional.

Tasks:

• What to do with interaction models where we have more than one vector for an entity/relation, such as e.g. TransH?
• Re-introduce slicing (In best case, on the generic level)
• Re-introduce regularization on generic level
• Fix R-GCN (or keep it broken for 📈 ☠️ Improve R-GCN #110 ?)
• Add regularizer/constrainer directly to modules, recursively search for modules having regularize and accumulate value.
• Update to reshape in the generic Interaction
• [ ] Update pipeline model composition bumped to 🪢 🤗 Expose interactions and representations via pipeline #163

Dependencies:

• Improve checking for abstract classes and applying post init hooks #137

[mberr]

TODO: What to do with interaction models where we have more than one vector for an entity/relation, such as e.g. TransH?

[mberr]

@cthoyt I used the interaction function abstraction to extract a common base class for DistMult, Complex and ER-MLP, since they all share that

• they have one vector representation for each entity and relation
• their interaction function is f(h, r, t) for appropriate f.

The class name is preliminary , and, for now, it resides in the Complex model's file. now it is in base.py.

[lvermue]

@mberr
What is the considered distinction of the interaction function and model?
Is that just because of R-GCN? Because right now it seems to me that the explicit model interaction function doesn't offer any generalization and is always "only" handled to the respective model class.

[mberr]

@mberr
What is the considered distinction of the interaction function and model?
Is that just because of R-GCN? Because right now it seems to me that the explicit model interaction function doesn't offer any generalization and is always "only" handled to the respective model class.

To quote from this paper 😉

Here, we define a KGEM as four components: an interactionmodel, a training approach, a loss function, and its usage of explicit inverse relations.

The idea is to separate storing the entity/relation representations from the interaction. This support use cases, where we want to use an interaction model as a component, rather than a stand-alone model, e.g. since we have features for entities, or another component which generates representations (e.g. based on GNNs, but we could also think about e.g. NLP models on the entities' labels, etc.).

In the long term this might also help to generalize e.g. the regularization choices (right now, each Model defines which parameters are regularized, and which not), or constraints, such as unit length, orthogonality, etc.

It does however also help for R-GCN 🙂

[mberr]

@cthoyt I extracted a few functional form interaction functions, cf. https://github.com/pykeen/pykeen/compare/a735593..f0c20d6

I wait for feedback before I continue with other interaction functions.

It is nice, since we e.g. could extract some common parts between distmult and complex in _normalize_terms_for_einsum.

[cthoyt]

Is this right? I thought SimplE's interaction function was

0.5 * (distmult_interaction(h, r, t) + dismult_interaction(t_inv, r_inf, h_inv)

The h and t get switched for the inverse one

[mberr]

@cthoyt did we finish the migration? If yes, we may close this one.

[cthoyt]

@mberr well, we have the major code for RGCN (might be fine now), literal models (already in the new PR), and all of the examples of the new-style models for all of the pre-existing uni-modal models

[cthoyt]

oops my bad, hand slipped

[cthoyt]

@mberr so all of the big parts have been migrated. If we ever want to start converting old-style models, we can refer to this, but it's finally time to let this one go.