ImportError: cannot import name 'BertOnlyMLMHead' from 'transformers'

[XuhuiZhou]

I installed the environment using
pip install -r requirements.txt

However, when I ran the example code,

python run.py \
    --task_name SST-2 \
    --data_dir data/k-shot/SST-2/16-42 \
    --overwrite_output_dir \
    --do_train \
    --do_eval \
    --do_predict \
    --evaluate_during_training \
    --model_name_or_path roberta-large \
    --few_shot_type prompt-demo \
    --num_k 16 \
    --max_steps 1000 \
    --eval_steps 100 \
    --per_device_train_batch_size 2 \
    --learning_rate 1e-5 \
    --num_train_epochs 0 \
    --output_dir result/tmp \
    --seed 42 \
    --template "*cls**sent_0*_It_was*mask*.*sep+*" \
    --mapping "{'0':'terrible','1':'great'}" \
    --num_sample 16 \

I encountered the following error:

Traceback (most recent call last):
  File "run.py", line 19, in <module>
    from src.models import BertForPromptFinetuning, RobertaForPromptFinetuning, resize_token_type_embeddings
  File "/home/xuhui/project/LM-BFF/src/models.py", line 6, in <module>
    from transformers import BertPreTrainedModel, BertForSequenceClassification, BertModel, BertOnlyMLMHead
ImportError: cannot import name 'BertOnlyMLMHead' from 'transformers'

I make sure that transformers==3.4.0 .
Any chance you may have more insight on this?

[XuhuiZhou]

I fixed this by using the following import commands in the src/models.py. But not sure whether that's the authors' intention.

from transformers.modeling_bert import BertPreTrainedModel, BertForSequenceClassification, BertModel, BertOnlyMLMHead
from transformers.modeling_roberta import RobertaForSequenceClassification, RobertaModel, RobertaLMHead, RobertaClassificationHead
from transformers.modeling_outputs import SequenceClassifierOutput

[gaotianyu1350]

Hi,

Maybe it's because you didn't pull the latest commit? Now in the repo the code is from transformers.modeling_bert instead of from transformers.

[XuhuiZhou]

Oh, great, thanks!