question_token_id=104

[yinzhangyue]

Hi, I have a quesition when I read the code, question_token_id=104, 104 is [MASK] in BERT vocab.txt, so it is still [MASK] token? This code is in finetuning/modeling.py.
class ModelWithQASSHead(BertPreTrainedModel): def __init__( self, config, replace_mask_with_question_token=False, mask_id=103, question_token_id=104, sep_id=102, initialize_new_qass=True, ):

[oriram]

Hi @yinzhangyue, [MASK] is 103 in BERT's vocab (zero-based counting). Our special [QUESTION] token is 104.

[yinzhangyue]

OK I get it. thx!