Adds a concrete finetuning example from a custom dataset

[alextrott16]

Adds a finetune_example directory in the train directory, which includes:

• A toy local dataset
• A preprocessing function
• A runnable YAML that puts it together for fine-tuning GPT2
• A README that explains each component

[samhavens]

This is going to be useful! I think everything is clear and merge-ready, except I think we need to address that arc_easy is under the CC-BY-SA license, which requires attribution. We can't put it in the JSONL file, since JSONL doesn't support comments. I think we are fine adding it to the finetune_example readme. The requirements are

You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.

Where "appropriate credit" means "you must provide the name of the creator and attribution parties, a copyright notice, a license notice, a disclaimer notice, and a link to the material."

So I'd add to the readme (fixing what I said about split to be right, if it we used a different split)

This work uses data from the AI2 Reasoning Challenge (ARC) 2018, created by and copyright AI2 and Aristo​​. The data consists of 7,787 science exam questions and is intended for non-commercial, research purposes only. We use the Easy Dev split, consisting of 570 questions and answer, which we do not modify. The original data is available at https://allenai.org/data/arc and is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

I am not sure why they gave it a commercial license but then added a noncommercial disclaimer.

[samhavens]

🤩