Question regarding the evaluation of video understanding

[cv-learner]

Thanks for sharing the code of this excellent work!

I have a question about the evaluation for video understanding. In TRAIN_AND_VALIDATE.md, the temporal understanding performance is evaluated using the questions from ChatUniVi/eval/questions/video_qa/generic_qa.json.

However, if I am not mistaken, shouldn't the evaluation of temporal understanding performance be based on the questions from ChatUniVi/eval/questions/video_qa/temporal_qa.json?

[jpthu17]

You are right, I think I made a serious mistake, I will immediately update the grade in the paper.

[jpthu17]

We've revised the temporal evaluation performance of video understanding, resulting in an actual model performance of 47.9 instead of the previously stated 57.8. We have also updated the evaluation script.

We sincerely apologize for any inconvenience our oversight may have caused you.