Why is the model constrained to fixed Image sizes?

[PardoAlejo]

Hi guys,
Thanks again for such a nice work. I want to ask you why is this assertion on the image size necessary? In case it is, any ideas on how to fix it? Thank you!

GroupViT/models/group_vit.py

Line 596 in 13b7861

assert H == self.img_size[0] and W == self.img_size[1], \

[xvjiarui]

Hi @PardoAlejo

Thanks for your kind words.
It's not necessarily asserted. It's just a sanity check.
You may comment it out.

[PardoAlejo]

Thanks for your reply, @xvjiarui.
Have you tried training the model with different image sizes? One would expect that larges image sizes or smaller patch sizes would give better segmentations.

[xvjiarui]

Hi @PardoAlejo

The smaller patch size will yield much more image tokens, and the self attention operator will require much more memory.
We didn't do that since we would like to pre-train with large batch size.