How to process the video that has more than 36 frames?

[jxiangli]

Thanks for your excellent work.
I have one qustion about the inference process. If the length of video is larger than 36, how to link the tracks from diffenrent clips?
looking forward to your reply.

[Epiphqny]

@jxiangli If the video is larger than 36, you could split the video into several clips, and link them based on the result of the overlapping frame using traditional post-processing.

[KainingYing]

@jxiangli If the video is larger than 36, you could split the video into several clips, and link them based on the result of the overlapping frame using traditional post-processing.
Thanks. By the way, the max length of Youtube VIS 2021 is 72 and 84 in train and val set respectively while the 2019 is 36.

[jxiangli]

Thanks for your reply~
I just started to research the VIS. Your means is that using post-processing like Kalman Filter and Hungarian algorithm to link the adjacent tracks, am I right?

[Epiphqny]

Thanks for your reply~
I just started to research the VIS. Your means is that using post-processing like Kalman Filter and Hungarian algorithm to link the adjacent tracks, am I right?

@jxiangli That's right.

[jxiangli]

Thank you very much!

[Epiphqny]

@jxiangli @noobying Thanks for pointing out it, I have not tried Youtube VIS 2021 before. In the original setting of VisTR, the number of instance queries is proportional to the number of input frames, and the instance queries are a fixed number of learned embedding. Therefore the number of input frames is fixed. But in row 3 of Table 1(b) of the paper, we also experiment with the instance-level queries that do not rely on the number of frames. In this way, the model could process a dynamic number of frames. But two many frames could also raise a memory issue, we leave it as future work.