Is there any option for fine-tuning mt5 models instead of training from scratch?

[farshadfiruzi]

Hi,
Thanks for the amazing simpleT5 package. I use the following script to train a mt5 model for summarization task.

from simplet5 import SimpleT5

model = SimpleT5()

model.from_pretrained(model_type="mt5", model_name="google/mt5-small")

model.train(train_df=train_df, eval_df=test_df, source_max_token_len=256, target_max_token_len=64, batch_size=8, max_epochs=3, use_gpu=True, outputdir = "outputs", early_stopping_patience_epochs = 0, )

When I run this code, training start from scratch. My question is that is there any flag to fine-tune the mt5 model instead of training from scratch?

[Shivanandroy]

model.train() finetunes the pre-trained model on your dataset. It doesn't train the model from scratch.

[farshadfiruzi]

Thank you so much.