TypeError in tensorflow/run_summarization.py

[bpraveenk]

Environment info

• transformers version: 4.11.0.dev0
• Platform: Linux-5.4.0-1055-azure-x86_64-with-glibc2.10
• Python version: 3.8.1
• PyTorch version (GPU?):
• Tensorflow version (GPU?): 2.5.0 (Yes)
• Using GPU in script?: Yes
• Using distributed or parallel set-up in script?: Distributed

Who can help:

@patrickvonplaten, @patil-suraj, @RocketKnight

Models: facebook/bart
datasets: xsum

• the official example scripts: (give details below)
  run_summarization.py
  relative path: examples/tensorflow/summarization

Steps to reproduce the behavior: (Note that --max_train_samples is optional)
python run_summarization.py --model_name_or_path facebook/bart-base --dataset_name xsum --dataset_config "3.0.0" --output_dir /tmp/tst-summarization --per_device_train_batch_size 4 --per_device_eval_batch_size 4 --num_train_epochs 3 --do_train --do_eval --max_train_samples 100

Error message

• INFO - main - Evaluation...
  0%| | 0/2833 [00:01<?, ?it/s]
  Traceback (most recent call last):
  File "run_summarization.py", line 663, in
  main()
  File "run_summarization.py", line 639, in main
  generated_tokens = model.generate(**batch)
  File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/pbodigut1/code/Users/pbodigut/transformers/v-4.10/transformers/src/transformers/generation_tf_utils.py", line 736, in generate
  output = self._generate_beam_search(
  File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/pbodigut1/code/Users/pbodigut/transformers/v-4.10/transformers/src/transformers/generation_tf_utils.py", line 1102, in _generate_beam_search
  model_inputs = self.prepare_inputs_for_generation(
  TypeError: prepare_inputs_for_generation() got multiple values for argument 'decoder_input_ids'

Expected behavior

Successfully run the evaluation step.

[harshitadd]

Hello,
Facing the same issue while using https://github.com/huggingface/transformers/tree/master/examples/tensorflow/translation for finetuning MarianMT's opus-mt-en-hi model.

Environment info:

transformers = '4.12.0.dev0'
tensorflow = 2.6.0
Python version: 3.8.5
Using GPU in script?: Yes

Stack Trace:

Traceback (most recent call last):
File "run_translation.py", line 626, in
main()
File "run_translation.py", line 610, in main
generated_tokens = model.generate( input_ids = input_ids, attention_mask = attention_mask, decoder_input_ids = decoder_input_ids, labels = labels)
File "/home/t-hdiddee/hf/lib/python3.8/site-packages/transformers/generation_tf_utils.py", line 736, in generate
output = self._generate_beam_search(
File "/home/t-hdiddee/hf/lib/python3.8/site-packages/transformers/generation_tf_utils.py", line 1102, in _generate_beam_search
model_inputs = self.prepare_inputs_for_generation(
TypeError: prepare_inputs_for_generation() got multiple values for argument 'decoder_input_ids'

[bpraveenk]

@Rocketknight1, wondering if there is any update on this issue.

[patrickvonplaten]

@Rocketknight1 - could you take a look? :-)

[bpraveenk]

I tested this simple fix, and it seems to work. In run_summarization.py, I added the line to remove decoder_input_ids from the batch dictionary (see below). My guess is that duplicate copies of decoder_input_ids are being passed to prepare_inputs_for_generation() function, after invoking model.generate(**batch).

if training_args.do_eval:
            logger.info("Evaluation...")
            for batch, labels in tqdm(
                tf_eval_dataset, total=len(eval_dataset) // training_args.per_device_eval_batch_size
            ):
             decoder_input_ids = batch.pop("decoder_input_ids", None)

@patrickvonplaten and @Rocketknight1 could you confirm if the change looks good to you and the fix doesn't break the BART model inference logic?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[Rocketknight1]

Hi @bpraveenk, firstly, sorry for the very long delay here! You're correct that supplying decoder_input_ids seems to break generate(). Your solution to pop() that key from the dictionary probably fixes the issue, but the underlying problem is that generate() simply doesn't know what to do with a decoder_input_ids input key. We're currently in the middle of entirely refactoring our TF generate() function for performance/bugfixing reasons, so I'll put that in the list of stuff to be investigated there, but I don't have a timeline on it unfortunately.

In the meantime, if you want to submit a PR with that change, please do! However, it will definitely only be a temporary workaround until we figure things out properly.

[bpraveenk]

Thank you for confirming the fix @Rocketknight1 . Could you point me to the PR submission instructions, I will submit the fix and I will add a comment that it is a temporary fix.

[Rocketknight1]

Hi @bpraveenk, thanks for the offer! We have instructions for contributors here.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.