What is Hard Negatives Mode?

[waynchi]

Hey, thanks for sharing your work.

1. Can you describe what your hard negatives mode is doing? I can't find any reference to it in the publication.
2. Did you train for 100 epochs? I found this in your config, but not in your publication.

Thanks!

[laituan245]

1. So basically except for COMETA, in-batch negative sampling is used by default. When hard_negatives_training is enabled, it also adds some top-ranked negative candidates (i.e., hard negatives) to the list of all negative candidates (please see the logic at this line 122). The list of hard negatives for each instance is automatically updated whenever the code does a full evaluation on the training set (see this line 70 and line 127).
2. Sorry I forgot to keep track of the exact number of epochs. But 100 is generally a good number. For COMETA or MedMentions (i.e., larger datasets), the number of epochs may be smaller (e.g., 25 or 50).