updated the wino_bias dataset

[JieyuZhao]

Updated the wino_bias.py script.

• updated the data_url
• added different configurations for different data splits
• added the coreference_cluster to the data features

[lhoestq]

I think we should only have the configurations "type1_pro", "type1_anti", "type2_pro" and "type2_anti".
Then each configuration can have train/dev/test splits. What do you think ?

[JieyuZhao]

Good point! Will update that. Thanks!

[lhoestq]

Hi @JieyuZhao ! Have you had a chance to add the different configurations ?
Thanks again for your help on this !

[JieyuZhao]

Hi @JieyuZhao ! Have you had a chance to add the different configurations ?
Thanks again for your help on this !

Hi @lhoestq Yes, I've updated the code. Now the configuration will have dev/test splits.

[lhoestq]

Cool thanks !
This looks perfect this way.

Now we just need to update the dataset_infos.json (it contains the metadata of the dataset) and add dummy data to be able to test this script automatically.

To update the dataset_infos.json you just need delete the current one at ./datasets/wino_biais/dataset_infos.json, and then run this command:

datasets-cli test ./datasets/wino_biais --save_infos --all_configs --ignore_verifications

To add the dummy data there's also a tool to add them automatically.
First delete the folder at ./datasets/wino_biais/dummy and then run

datasets-cli dummy_data ./datasets/wino_biais --auto_generate --match_text_files "*conll" --n_lines 15

Let me know if you have questions :)
Also don't forget to run make style to format the code properly.

[JieyuZhao]

Cool thanks !
This looks perfect this way.

Now we just need to update the dataset_infos.json (it contains the metadata of the dataset) and add dummy data to be able to test this script automatically.

To update the dataset_infos.json you just need delete the current one at ./datasets/wino_biais/dataset_infos.json, and then run this command:

datasets-cli test ./datasets/wino_biais --save_infos --all_configs --ignore_verifications

To add the dummy data there's also a tool to add them automatically.
First delete the folder at ./datasets/wino_biais/dummy and then run

datasets-cli dummy_data ./datasets/wino_biais --auto_generate --match_text_files "*conll" --n_lines 15

Let me know if you have questions :)
Also don't forget to run make style to format the code properly.

Thanks for the instruction! I've updated the metadata and the dummy data and also do the formatting. Please let me know if more is needed. :)

[lhoestq]

Thank you !