ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds Transformers Translation Tutorial Repro

[SoyGema]

System Info

Context

Hello There!
First and foremost, congrats for Transformers Translation tutorial. 👍
It serves as a Spark for building english-to-many translation languages models!
I´m following it along with TF mostly reproducing it in a jupyter Notebook with TF for mac with GPU enabled
Using the following dependency versions.

tensorflow-macos==2.9.0
tensorflow-metal==0.5.0
transformers ==4.29.2

* NOTE : tensorflow-macos dependencies are fixed for ensuring GPU training

Who can help?

@ArthurZucker @younesbelkada
@gante maybe?

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

Issue Description

Im finding the following error when fitting the model for finetunning a model coming from TFAutoModelForSeq2SeqLM autoclass

with tf.device('/device:GPU:0'):
    model.fit(x=tf_train_set, validation_data=tf_test_set, epochs=1, callbacks= callbacks ) 

It is returning

ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds
        
        
        Call arguments received by layer "decoder" (type TFT5MainLayer):
          • self=None
          • input_ids=None
          • attention_mask=None
          • encoder_hidden_states=tf.Tensor(shape=(32, 96, 512), dtype=float32)
          • encoder_attention_mask=tf.Tensor(shape=(32, 96), dtype=int32)
          • inputs_embeds=None
          • head_mask=None
          • encoder_head_mask=None
          • past_key_values=None
          • use_cache=True
          • output_attentions=False
          • output_hidden_states=False
          • return_dict=True
          • training=False
    
    
    Call arguments received by layer "tft5_for_conditional_generation" (type TFT5ForConditionalGeneration):
      • self={'input_ids': 'tf.Tensor(shape=(32, 96), dtype=int64)', 'attention_mask': 'tf.Tensor(shape=(32, 96), dtype=int64)'}
      • input_ids=None
      • attention_mask=None
      • decoder_input_ids=None
      • decoder_attention_mask=None
      • head_mask=None
      • decoder_head_mask=None
      • encoder_outputs=None
      • past_key_values=None
      • inputs_embeds=None
      • decoder_inputs_embeds=None
      • labels=None
      • use_cache=None
      • output_attentions=None
      • output_hidden_states=None
      • return_dict=None
      • training=False

Backtrace

Tried:

• Remove callbacks : The model is trained, but of course not loaded into the Hub, nor the metrics computed

• Followed Loading from AutoModel gives ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds #16234 , this comment and ensured that Im using AutoTokenizer. This glimpsed that this could be related to TFAutoModelForSeq2SeqLM .

model = TFAutoModelForSeq2SeqLM.from_pretrained(checkpoint)

Seems to be working correctly. Therefore I assume that the pre-trained model is loaded

• Also followed PushToHubCallback is hanging on the training completion #21116 and added save_strategy=no argument in PushToCallBack , but the error persisted

Expected behavior

Model trained should be uploaded to the Hub.
The folder appears empty , there is an error

Hypothesis

At this point, what Im guessing is that once I load the model I shall redefine the verbose error trace?
Any help please of how to do this ? :) or how can I fix it ? Do I have to define a specific Trainer ? Any idea of where I can find this in docs?

[gante]

Hey @SoyGema 👋

From your exception, I believe the issue is at the data preparation stage -- it is pretty much complaining that your dataset has no labels. Have you followed the data preprocessing steps described here?

[SoyGema]

Hello there @gante ! Thanks for your quick response and help !
I really appreciate it . 🥇
I´ve uploaded the notebook here . As far as I can understand (let me know if Im missing something here ), Im using the preprocessing function.

In fact, the _tokenized_books (cell 16) returns something in the form of

DatasetDict({
    train: Dataset({
        features: ['id', 'translation', 'input_ids', 'attention_mask', 'labels'],
        num_rows: 1123
    })
    test: Dataset({
        features: ['id', 'translation', 'input_ids', 'attention_mask', 'labels'],
        num_rows: 281
    })
})

And data_collator (cell 19) returns something like

DataCollatorForSeq2Seq(tokenizer=T5Tokenizer(name_or_path='t5-small', vocab_size=32100, model_max_length=512, is_fast=False, padding_side='right', truncation_side='right', special_tokens={'eos_token': '</s>', 'unk_token': '<unk>', 'pad_token': '<pad>', 'additional_special_tokens': ['<extra_id_0>', .....

Am I missing something from the video that should be in code ?
for quick testing purposes, Im with pt_to_en dataset, that seems to have same characteristics. I've checked that tokenized_books function returns the same data structure type in pt_to_en that in fr_to_en dataset

My apologies in advance for the extremely notebook verbose code regarding GPU low level operation use. I am trying to optimize for that therefore all trace.

Thanks so so much for your time on this
Happy if you can point me on the right direction! 👍

[gante]

Hey @SoyGema 👋

Your KerasMetricCallback was missing predict_with_generate=True -- metrics that rely on text generation must pass this flag, as generating text is different from a model forward pass. It should become metric_callback = KerasMetricCallback(metric_fn=compute_metrics, eval_dataset=tf_test_set, predict_with_generate=True)

For future reference in case you encounter further bugs, have a look at our complete translation example: https://github.com/huggingface/transformers/blob/main/examples/tensorflow/translation/run_translation.py

[SoyGema]

Hello there @gante 👋

Thanks for the reference. I'm definetly having this as a north script and also using it !
Been thinking about how to structure this exploration and also indexing the roadblocks/bugs/solutions so other users can benefit from it .

I'm closing this issue (as it is solved but other arised )and probably open another ones in my own repo as it goes so issues are unitary-structured . Hope this makes sense. Hope I can take it from there and not disturb you!

Thanks again!

[SoyGema]

Just for Reproducibility. If someone wants to go through the script example. Documentation about flag configuration and more can be found here