Evaluation scheme for larger KBs

[vardaan123]

In the paper, section 5.3, it is mentioned that for the entire Freebase KB, you use a modified evaluation strategy in which

we use only 2000 negative triplets; 1000 sampled uniformly from the entire set of negative
samples and 1000 sampled proportionally to the degree of the corrupted entities;

Is this evaluation supported in the dgl-ke library? If so, which arguments can be used to enable this?

[classicsong]

you can use this argument --neg_sample_size_eval=1000 and neg_deg_sample_eval=True

[vardaan123]

Thanks for your response!

[vardaan123]

Hi, for evaluation, I understand that the corrupted entities are a mix of uniformly sampled entities and those sampled based on degree (if I use --neg_sample_size_eval=1000 and neg_deg_sample_eval=True). Is there a way if I want to use just degree-based sampling and no uniformly sampled negative triples for evaluation?

[classicsong]

Currently, it does not support certain sampling method. You have to implement it your self.