Question about parameter tuning

[lxa9867]

Hi bowen,

Thank you for sharing such a great work.

I have a question about the parameter for training on new dataset for panoptic segmentation task. In the new dataset, we have less objects in each image (maybe 1-5). What parameter do you think is the most important to conduct this adaptation? Any advice is really appreciated.

The situation is that I found the final mask and dice loss are close to 0.1 which is somehow smaller than it in coco panoptic training (about 0.3)? I wonder is there any normalization in the code to make the loss small when object number is small? I think no?

Another random question: do you use a large batchsize of 64 because of the poor label quality of coco? If I change it to 16 will that have a large difference (I ask this because of the panoptic-deeplab pytorch version using 16 while paper using 64 also)?

Look forward to your reply.

[bowenc0221]

The loss is normalized by the number of GT boxes:

MaskFormer/mask_former/modeling/criterion.py

Line 165 in 232bd1b

num_masks = sum(len(t["labels"]) for t in targets)

We use batchsize of 64 following DETR, and we have not tried using a smaller batchsize in this work.

[lxa9867]

Is the box only used to normalize? I think you use the dice of mask and class prob to conduct matching between gt and preds?

[bowenc0221]

In bipartite matching, we calculate all pairwise losses, so there is no normalization needed in matching.

[lxa9867]

Sorry for the confusion. My question is do you use the box for matching or as part of the loss function?

[bowenc0221]

Boxes are never used. Please ignore the name in comment (copied from DETR code).

[lxa9867]

I see, thank you for your answers.