I guess this is a bug...

[beyondguo]

nlpaug/nlpaug/model/lang_models/lambada.py

Line 58 in f34e7cd

input_text = 'label_{} {}'.format(label, self.sep_token)

Orig:

input_text = 'label_{} {}'.format(label, self.sep_token)

I believe it should be:

input_text = '{} {}'.format(label, self.sep_token)

Because the original input format mismatches the format in the pre-processing script:

def prepare_mlm_data(labels, texts, output_file_path, sep_token):
	if not os.path.exists(output_file_path):
		os.makedirs(output_file_path)
	with open(os.path.join(output_file_path, 'mlm_data.txt'), 'w') as f:
		for label, text in zip(labels, texts):
			f.write(' '.join([str(label), sep_token, text]) + '\n')

I have tried to generate samples by your code, but the generated text is very strange. After I made the changes above, the results become normal.

[makcedward]

You are right. This fix will be included in the coming release