Evaluation setting - Tracking forward & backward?

[npmhung]

I would like to clarify the evaluation setting.

Because the query points can be sampled any where within the video (not just in the first frame), do we have to track them backward in time or just need to track them forward?

For example, if the query point is sample at frame T, do we have to find its position in frames 0->(T-1), or just need to track it in (T+1)->Max_Frame?

[m43]

Hej, in the first query mode, only the future time steps need to be tracked. This setting is particularly relevant in an online context. However, in the strided mode, all frames are evaluated. For a more detailed understanding, you can refer to the evaluation code snippet below or check appendix section H of the the TAP-Vid paper. Hope this helps!

https://github.com/deepmind/tapnet/blob/7b3a336072c5a84797fa4d51acae16a38d613566/evaluation_datasets.py#L105-L113