Some questions about ablation study

[JerryX1110]

I am curious about the performance under the following setting. Have you tried them before?

1. STCN w/o V.S. W/ Top-k filtering
2. STCN V.S. STM with the same training strategy (I find that the pretraining of STCN maybe uses more data and some augmentations.

[JerryX1110]

What's more, I wanna know if the ablation study between "Every 5th + Last" and "Every 5th only" is predicted by the same model under different memory management strategies? Or these two models are trained separately?

[hkchengrex]

1. You can try these by setting top_k=None in inference. The same model can be used. Top-k gives a nice performance boost as I remembered -- you can also compare that result with MiVOS, which is essentially STM+top-k.
2. STM does not have the official training code so that's difficult to replicate. I used the exact same data/augmentation as my previous work which has a very similar structure and performance as STM (if you disregard top-k). That should be evident that the data strategy performs similarly as STM.
3. Same model.

[JerryX1110]

Got it, thanks!

[JerryX1110]

I think maybe "Every 5th + Last" is worse than "Every 5th only" probably because you sample your video clip in an interval that is >=5. Do you agree with that?

[JerryX1110]

I mean the sampling strategy in the training stage.

[hkchengrex]

No.

1. The minimum distance between sampled frames is 1 -- you are probably mixing up the maximum distance and the minimum distance.
2. The same sampling is used in our STM training, but "every 5th + last" is still better.

[JerryX1110]

Thanks!