XNLI Processor alignment issue

[tonytan48]

In the preprocessing script of XNLI: xtreme/third_party/processors/xnli.py. Line 42-43. The first line of xnli processor will be skipped. I noted that this processor is inherited from transformers processor class, this is useful for the raw XNLI 1.0 dataset, as it contains header. However, xtreme preprocessing script will first remove the original XNLI datasets' header: like premise,label,etc. For the preprocessed xnli dataset {split}-{lang} like dev-en.tsv, the first actual example will be skipped. Maybe the line 42-43 of xtreme/third_party/processors/xnli.py can be deleted as the preprocessing script already removed the header of XNLI.

[melvinjosej]

Thanks @tonytan48 for raising this. I checked this and you're right, we should be deleting L42-43 in .../processors/xnli.py . Can you please delete those two lines and send a PR? I'll approve it.

[tonytan48]

Aight thanks