Erroneous code

[winitzki]

BERT-pytorch/bert_pytorch/dataset/dataset.py

Lines 53 to 62 in 7efd2b5

	if prob < prob * 0.8:
	tokens[i] = self.vocab.mask_index
	# 10% randomly change token to random token
	elif prob * 0.8 <= prob < prob * 0.9:
	tokens[i] = random.randrange(len(self.vocab))
	# 10% randomly change token to current token
	elif prob >= prob * 0.9:
	tokens[i] = self.vocab.stoi.get(token, self.vocab.unk_index)

This code is incorrect - it will always go into the last if clause. For instance, prob < prob * 0.8 is never true.

[codertimo]

fixed by #22 duplicated with #13