Pretraining with CutLER

[VGrondin]

Hello, reading your paper was pretty interesting and insightful.
I was wondering how well an object detector model such as ViTDet can benefit by pretraining with CutLER?
For instance, from the ViTDet paper, the authors acheive 55.6 APbox and 49.2. APmask (table 5 in Exploring Plain Vision Transformer Backbones for Object Detection), so is it possible to pretrain a ViTDet with CutLER and finetune it in a supervised learning way on COCO to improve detection results?

Thanks again for the great paper.

[frank-xwang]

Hi @VGrondin, I believe that using ViTDet pretrained with CutLER could potentially improve performance for semi-/fully supervised learning. I would be interested to hear about any updates you may have on any observed performance gains. Thank you!

[VGrondin]

Hi @frank-xwang, thanks for the information. My interest in CutLER mostly lies in the performance improvement on a custom dataset of forest images. I will let you know how it goes, I prepared the pseudo masks using maskcut with a mae backbone pretrained on 100k forest images (example of pseudo mask generated):

Now I am trying to load vitdet config in cutler, but I am having some compatibility issues between the .yaml config and the recent .py config that vitdet uses. I will let you know how it goes once I find a way to resolve this issue.