How to adapt run_pretraining.sh for a different set of training data?

[gabriead]

Hi guys, first of all great job on the model! I would like to adapt the model to a different type of domain data. I am having a hard time to understand how I have to adapt the run_pretraining.sh to my local machine. Could you include a description of where the script is getting it's data (where is gs://t5-data/pretrained_models/ ?) and how to configure the parameters (e.g. what is in dataset.gin ?) to a different project. Is the model pretrained on all of those datasets (see screenshot) before it's finetuned on the custom google datasets? Or what is their purpose?

[danyaljj]

These scripts are using T5 for pre-training.
You'd need to consult with this Github repository for details:
https://github.com/google-research/text-to-text-transfer-transformer/

[gabriead]

Ok thank you. As I understand finetuning a T5 requires triples of training data: "name of the task to be trained" + "question" + "answer". If I am not mistaken you are using pairs consisting of "question" + "answer". Could you explain how the model knows what's it suppose to learn without the prepended task description?