A bug in dodrio-data-gen.py

[Yukyin]

Hello, authors, It's excellent work! And I have a little bug unsolved when I run dodrio-data-gen.py.
The bug seems here
my_loss,my_logit,attentions=my_model(tokens,attention_mask=masks,labels=labels.long(),output_attentions=True)

The details are shown as follows:

Using device: cuda
Reusing dataset glue (/root/.cache/huggingface/datasets/glue/sst2/1.0.0/7c99657241149a24692c402a5c3f34d4c9f1df5ac2e4c3759fadea38f6cb29c4)
2. Extracting Attention Weights and Gradients...
Some weights of the model checkpoint at bert-base-uncased were not used when initializing MyBertForSequenceClassification: ['cls.predictions.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.seq_relationship.weight', 'cls.predictions.decoder.weight', 'cls.predictions.transform.dense.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.bias']
- This IS expected if you are initializing MyBertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing MyBertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of MyBertForSequenceClassification were not initialized from the model checkpoint at bert-base-uncased and are newly initialized: ['classifier.bias', 'classifier.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForSequenceClassification: ['cls.predictions.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.seq_relationship.weight', 'cls.predictions.decoder.weight', 'cls.predictions.transform.dense.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.bias']
- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-base-uncased and are newly initialized: ['classifier.bias', 'classifier.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
  0%|                                                                    | 0/127 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/yukyin/anaconda3/envs/exp381/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/yukyin/anaconda3/envs/exp381/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/root/.vscode-server/extensions/ms-python.python-2022.4.1/pythonFiles/lib/python/debugpy/__main__.py", line 45, in <module>
    cli.main()
  File "/root/.vscode-server/extensions/ms-python.python-2022.4.1/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py", line 444, in main
    run()
  File "/root/.vscode-server/extensions/ms-python.python-2022.4.1/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py", line 285, in run_file
    runpy.run_path(target_as_str, run_name=compat.force_str("__main__"))
  File "/home/yukyin/anaconda3/envs/exp381/lib/python3.8/runpy.py", line 265, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/home/yukyin/anaconda3/envs/exp381/lib/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/home/yukyin/anaconda3/envs/exp381/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/yukyin/PycharmProjects/test/xiaoyanghua_new/humor/dodrio/data-generation/dodrio-data-gen.py", line 966, in <module>
    output = my_model(input_ids=tokens
  File "/home/yukyin/anaconda3/envs/exp381/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/yukyin/anaconda3/envs/exp381/lib/python3.8/site-packages/transformers/models/bert/modeling_bert.py", line 1502, in forward
    outputs = self.bert(
  File "/home/yukyin/anaconda3/envs/exp381/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/yukyin/anaconda3/envs/exp381/lib/python3.8/site-packages/transformers/models/bert/modeling_bert.py", line 971, in forward
    encoder_outputs = self.encoder(
  File "/home/yukyin/anaconda3/envs/exp381/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/yukyin/anaconda3/envs/exp381/lib/python3.8/site-packages/transformers/models/bert/modeling_bert.py", line 568, in forward
    layer_outputs = layer_module(
  File "/home/yukyin/anaconda3/envs/exp381/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
TypeError: forward() takes from 2 to 7 positional arguments but 8 were given

I have tried different versions of transformers,but it doesn't work.

[xiaohk]

Sorry for the late reply! Perhaps try to install transformers==3.3.1?

Here is the our conda environment:

name: nlp
channels:
  - pytorch
  - conda-forge
  - defaults
dependencies:
  - _libgcc_mutex=0.1=main
  - attrs=20.2.0=pyh9f0ad1d_0
  - backcall=0.2.0=py_0
  - blas=1.0=mkl
  - bleach=3.2.1=pyh9f0ad1d_0
  - ca-certificates=2020.6.20=hecda079_0
  - certifi=2020.6.20=py37hc8dfbb8_0
  - cudatoolkit=10.0.130=0
  - decorator=4.4.2=py_0
  - defusedxml=0.6.0=py_0
  - entrypoints=0.3=py37hc8dfbb8_1001
  - freetype=2.10.2=h5ab3b9f_0
  - importlib-metadata=2.0.0=py37hc8dfbb8_0
  - importlib_metadata=2.0.0=0
  - intel-openmp=2020.2=254
  - ipykernel=5.3.4=py37h5ca1d4c_0
  - ipython=7.18.1=py37h5ca1d4c_0
  - ipython_genutils=0.2.0=py37_0
  - ipywidgets=7.5.1=pyh9f0ad1d_1
  - jedi=0.17.2=py37_0
  - jinja2=2.11.2=pyh9f0ad1d_0
  - jpeg=9b=h024ee3a_2
  - jsonschema=3.2.0=py37hc8dfbb8_1
  - jupyter_client=6.1.7=py_0
  - jupyter_core=4.6.3=py37_0
  - lcms2=2.11=h396b838_0
  - ld_impl_linux-64=2.33.1=h53a641e_7
  - libedit=3.1.20191231=h14c3975_1
  - libffi=3.3=he6710b0_2
  - libgcc-ng=9.1.0=hdf63c60_0
  - libpng=1.6.37=hbc83047_0
  - libsodium=1.0.18=h7b6447c_0
  - libstdcxx-ng=9.1.0=hdf63c60_0
  - libtiff=4.1.0=h2733197_1
  - lz4-c=1.9.2=he6710b0_1
  - markupsafe=1.1.1=py37h8f50634_1
  - mistune=0.8.4=py37h8f50634_1001
  - mkl=2020.2=256
  - mkl-service=2.3.0=py37he904b0f_0
  - mkl_fft=1.2.0=py37h23d657b_0
  - mkl_random=1.1.1=py37h0573a6f_0
  - nbformat=5.0.7=py_0
  - ncurses=6.2=he6710b0_1
  - ninja=1.10.1=py37hfd86e86_0
  - numpy=1.19.1=py37hbc911f0_0
  - numpy-base=1.19.1=py37hfa32c7d_0
  - olefile=0.46=py37_0
  - openssl=1.1.1h=h516909a_0
  - packaging=20.4=pyh9f0ad1d_0
  - pandoc=2.10.1=h516909a_0
  - pexpect=4.8.0=py37_1
  - pickleshare=0.7.5=py37_1001
  - pillow=7.2.0=py37hb39fc2d_0
  - pip=20.2.3=py37_0
  - prometheus_client=0.8.0=pyh9f0ad1d_0
  - prompt-toolkit=3.0.7=py_0
  - pygments=2.7.1=py_0
  - pyparsing=2.4.7=pyh9f0ad1d_0
  - pyrsistent=0.17.3=py37h8f50634_0
  - python=3.7.9=h7579374_0
  - python-dateutil=2.8.1=py_0
  - python_abi=3.7=1_cp37m
  - pytorch=1.4.0=py3.7_cuda10.0.130_cudnn7.6.3_0
  - readline=8.0=h7b6447c_0
  - send2trash=1.5.0=py_0
  - setuptools=49.6.0=py37_1
  - six=1.15.0=py_0
  - sqlite=3.33.0=h62c20be_0
  - terminado=0.9.1=py37hc8dfbb8_0
  - testpath=0.4.4=py_0
  - tk=8.6.10=hbc83047_0
  - torchvision=0.5.0=py37_cu100
  - tornado=6.0.4=py37h7b6447c_1
  - traitlets=5.0.4=py_0
  - wcwidth=0.2.5=py_0
  - wheel=0.35.1=py_0
  - widgetsnbextension=3.5.1=py37hc8dfbb8_1
  - xz=5.2.5=h7b6447c_0
  - zeromq=4.3.2=he6710b0_3
  - zlib=1.2.11=h7b6447c_3
  - zstd=1.4.5=h9ceee32_0
  - pip:
    - argon2-cffi==20.1.0
    - async-generator==1.10
    - captum==0.2.0
    - cffi==1.14.3
    - chardet==3.0.4
    - click==7.1.2
    - cycler==0.10.0
    - datasets==1.4.1
    - dill==0.3.3
    - filelock==3.0.12
    - fsspec==0.8.7
    - huggingface-hub==0.0.2
    - idna==2.10
    - install==1.3.4
    - ipython-genutils==0.2.0
    - jenkspy==0.2.0
    - joblib==0.17.0
    - json5==0.9.5
    - jupyterlab==2.2.8
    - jupyterlab-pygments==0.1.2
    - jupyterlab-server==1.2.0
    - kiwisolver==1.2.0
    - matplotlib==3.3.2
    - multiprocess==0.70.11.1
    - nbclient==0.5.0
    - nbconvert==6.0.6
    - nest-asyncio==1.4.1
    - networkx==2.5
    - notebook==6.1.4
    - opencv-python==4.4.0.46
    - pandas==1.1.3
    - pandocfilters==1.4.2
    - parso==0.7.1
    - protobuf==3.15.5
    - ptyprocess==0.6.0
    - pyarrow==3.0.0
    - pycparser==2.20
    - pytz==2020.1
    - pyzmq==19.0.2
    - regex==2020.9.27
    - requests==2.24.0
    - sacremoses==0.0.43
    - scikit-learn==0.23.2
    - scipy==1.5.2
    - seaborn==0.11.0
    - sentencepiece==0.1.91
    - sklearn==0.0
    - stanza==1.2
    - threadpoolctl==2.1.0
    - tokenizers==0.8.1rc2
    - tqdm==4.49.0
    - transformers==3.3.1
    - urllib3==1.25.10
    - webencodings==0.5.1
    - xxhash==2.0.0
    - zipp==3.2.0

I will close the issue for now. Let me know if it does not fix the issue. Thanks!