Question about random sampling.

[SongRb]

BERT-pytorch/bert_pytorch/dataset/dataset.py

Lines 50 to 64 in 7efd2b5

	prob = random.random()
	if prob < 0.15:
	# 80% randomly change token to make token
	if prob < prob * 0.8:
	tokens[i] = self.vocab.mask_index
	# 10% randomly change token to random token
	elif prob * 0.8 <= prob < prob * 0.9:
	tokens[i] = random.randrange(len(self.vocab))
	# 10% randomly change token to current token
	elif prob >= prob * 0.9:
	tokens[i] = self.vocab.stoi.get(token, self.vocab.unk_index)
	output_label.append(self.vocab.stoi.get(token, self.vocab.unk_index))

Well, seems random.random() always returns a positive number, so prob >= prob * 0.9 will always be true?

[codertimo]

Haha your right it seems else is more efficient. thank you for your comment 👀

[leon-cas]

if prob < prob * 0.8: always False?
elif prob * 0.8 <= prob < prob * 0.9: always False?

[artemisart]

I also think these conditions are still wrong, I'm sending a PR