About shifted label distribution

[sunxiaojie99]

Hi~I have just read your paper and have a question. In this paper, the distribution of the relation_label between the train set and the test is different, so if we test directly on the test set, we can not test the ability of the model trained on the train set, So this paper puts forward two methods: threshold and bias.
But I don't think this problem is the essence of distant supervised relation extraction. Different from label noise, this problem may be the deficiency of dataset publishers when they make datasets. Even if the test set is labeled manually, we still can adjust the distribution of the test set artificially according to the distribution of relation_label in the train set.
So I can think that the method proposed in this paper is only applicable to the premise that the distribution of the existing train set and test set is different, and I don't want to change the distribution of any of the datasets. And how can we really evaluate the performance of the model trained on the train set?