🔬🔁 Evaluation loop

[mberr]

This PR adds an evaluation loop based upon torch's data loaders, and delegates the automatic batch size optimization to torch-max-mem. It also brings support for relation prediction evaluation.

[mberr]

target-specific evaluation datasets have been moved to a new branch, cf. evaluation-loop-2, as they require additional changes to the evaluation to enable separate, independently optimized batch sizes for different targets.

[cthoyt]

@mberr lgtm but would be nice to get a second reviewer

[migalkin]

Tried the branch on the ILPC codebase - it works, can reproduce the numbers 👍
My only small wish (which is more about result tracking and hot this PR) is that the default console result tracker dumps all 100+ metrics in the console after each eval step and it's kinda too much, I'd like to have a way to set up seeing maybe only 1-3 representative metrics like realistic hits@10 and inverse harmonic mean rank

[mberr]

Tried the branch on the ILPC codebase - it works, can reproduce the numbers 👍 My only small wish (which is more about result tracking and hot this PR) is that the default console result tracker dumps all 100+ metrics in the console after each eval step and it's kinda too much, I'd like to have a way to set up seeing maybe only 1-3 representative metrics like realistic hits@10 and inverse harmonic mean rank

Did you know about the metric_filter parameter of the ConsoleTracker? 😇 e.g.

from pykeen.pipeline import pipeline

result = pipeline(
    dataset="nations",
    model="mure",
    result_tracker="console",
    result_tracker_kwargs=dict(metric_filter=r"both.realistic.(hits_at_10|inverse_harmonic_mean_rank)"),
)

will only print realistic H@10 and MRR averaged over head & tail.