[MASK] is appended to the end of questions instead of [QUESTION] when finetuning

[Liangtaiwan]

Hi,

I found that in qass_preprocess.py.
You append [MASK] to the end of questions instead of [QUESTION]

splinter/finetuning/qass_preprocess.py

Line 8 in 55866bb

MASK_TOKEN = "[MASK]"

splinter/finetuning/qass_preprocess.py

Lines 18 to 26 in 55866bb

	def insert_mask_to_qa_end(data_point):
	transformed_data_point = data_point.copy()
	qas = []
	for qa in transformed_data_point["qas"]:
	qa = qa.copy()
	qa["question"] += f" {MASK_TOKEN}."
	qas.append(qa)
	transformed_data_point["qas"] = qas
	return transformed_data_point

Is there a difference between implementation and paper?
or this is a bug?

[oriram]

Hi @Liangtaiwan, this is not a bug.
The mask token is replaced with a question token in the model itself.
This is not the most elegant solution and will be changed when the model is integrated in HuggingFace's transformers library.
See these lines:

splinter/finetuning/modeling.py

Lines 113 to 115 in 55866bb

	if self.replace_mask_with_question_token:
	input_ids = input_ids.clone()
	input_ids[input_ids == self.mask_id] = self.question_token_id

[Liangtaiwan]

Hi @oriram,
Thanks :)
just found that I did not see

This will add a [MASK] token to each question in the training data, which will later be replaced by a [QUESTION] token automatically by the QASS layer implementation.