You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Thanks for your interest in our work.
As shown in this Line, there is ~indices_replaced to make sure only 10% are replaced by random word.
You can reorder this line as: indices_random = torch.bernoulli(torch.full(input_ids.shape, 0.5)).bool() & ~indices_replaced & masked_indices
Please let me know if you might have any other questions.
Hi,
Here is the code you use to replace a token with a random word. Is it correct to use 0.5 as the parameter here?
Thank you for your answer.
The text was updated successfully, but these errors were encountered: