Bad zero-shot results on TVQA

[fake-warrior8]

Hi, I ran the zero-shot result on TVQA dataset with the given zero-shot checkpoint frozenbilm.pth and the given TVQA video features clipvitl14.pth. I also used the microsoft/deberta-v2-xlarge checkpoint. However, I got the val acc 31.59 instead of the reported 59.7.

[antoyang]

Are you loading the subtitles correctly? The text-only model with subtitles should already get better accuracy without further training. You may check that the perf looks ok on other datasets too.

[fake-warrior8]

Are you loading the subtitles correctly? The text-only model with subtitles should already get better accuracy without further training. You may check that the perf looks ok on other datasets too.

Thank you for your reply. I found that I loaded an empty subtitles. I used the dataset providers to download the subtitles and got an empty subtitle file. When I replace the subtitles to your given Google drive subtitles, I got the right results.