Wrong contents in gold standard

[tfmorris]

When I look at the contents of https://github.com/dkpro/dkpro-c4corpus/blob/master/dkpro-c4corpus-boilerplate/BoilerplateEvaluationOnCleanEval/CleanEvalGoldStandard/103.txt the header says that it is 104.txt and the contents match the contents of 104, even though the file name is 103

[OmniaZayed]

Hi Tom,

This gold standard dataset is downloaded from CLEANEVAL homepage without applying any changes. So this header/content mix of the file 103.txt is originated from CLEANEVAL itself.
Thanks,
Best regards,
Omnia

[tfmorris]

Thanks for the quick response. Any idea how widespread the corruption is? Obviously having completely unrelated gold standard data is going to render the scores invalid.

[OmniaZayed]

You are welcome.
I am not sure if there is other corrupted files or not. But if you opened
103.txt with notepad++ (kindly check the attached picture) you will see
some strange lines at the beginning of the file which does not occur in
some other files that I have checked so far.

[tfmorris]

I looked into it a little further using the original data set and 103.txt is a tar file made up of all the other files. My guess is that someone ran a command like tar cvf *.txt which the shell expanded to tar cvf 103.txt 104.txt 105.txt ... overwriting the original 103.txt with a tar made up of all the other files.

I'd recommend just deleting it since it contains no useful information. The only way to recover the original file would be to contact the authors.

[OmniaZayed]

Yes, you are right. Thanks a lot for the clarification.

[habernal]

Has to be documented in #4

[tfmorris]

Just discovered that there's another repo with the GoldStandard files in it in case any more problems turn up: https://github.com/ppke-nlpg/boilerplateResults/tree/master/cleanEvalResults/GoldStandard