Fix for Webgpt NaN loss

[shahules786]

Webgpt dataset was giving NaN loss/metrics during RM training. This was due to the presence of samples in the dataset with empty string answers.
for example,
{'question': {'dataset': 'arc-challenge', 'id': 'Mercury_7228550', 'full_text': 'How many basic units of information in a DNA molecule are required to encode a single amino acid?\nA. 1\nB. 2\nC. 3\nD. 4'}, 'quotes_0': {'title': [], 'extract': []}, 'answer_0': '', 'tokens_0': {'prefix': [2437, 867, 4096, 4991, 286, 1321, 287, 257, 7446, 27756, 389, 2672, 284, 37773, 257, 2060, 23206, 7408, 30, 198, 32, 13, 352, 198, 33, 13, 362, 198, 34, 13, 513, 198, 35, 13, 604, 48366], 'completion': [48366]}, 'score_0': 0.0, 'quotes_1': {'title': [], 'extract': []}, 'answer_1': '', 'tokens_1': {'prefix': [2437, 867, 4096, 4991, 286, 1321, 287, 257, 7446, 27756, 389, 2672, 284, 37773, 257, 2060, 23206, 7408, 30, 198, 32, 13, 352, 198, 33, 13, 362, 198, 34, 13, 513, 198, 35, 13, 604, 48366], 'completion': [48366]}, 'score_1': 0.0}

fixes : #2439

[sanagno]

Thanks a lot for that, good catch!

[andreaskoepf]

thanks for debugging the NaNs! :-)