Could not figure how to setup datasets to run the scripts

[afonso-sousa]

Hi there. Congrats on the great work. I would like to try running your code. I started with the prepare_data.sh script but cannot figure out how to setup the datasets. I set the raw_datasets as data and processed data as a new empty folder. Is this it? For me, the script computes the bleu score and freezes. Besides, how did you get to the *.csv.gz files? Are these the full source datasets? Can you publish how did you process the data or where did you get it from? Why does the prepare_data.sh script call evaluate.py? Sorry if my question seem too trivial, just trying to understand how to reproduce the data processing for now. Thanks for any help you can provide.