Linear segmentation evaluation on ADE20k

[Adrien987k]

Hi,

I am trying to reproduce the linear segmentation results obtained with the ViT-B IBOT pretrained model, which performs at 38.3 mIoU according to the paper.

With this model, and the config file provided in:

ibot/evaluation/semantic_segmentation/configs/linear/vit_base_512_ade20k_160k.py

I only reach ~18mIoU on ADE20K.
I saw that the command in the README change the learning rate and normalize the output so I tried with:

model.backbone.out_with_norm=true  optimizer.lr=8e-4

and I got ~20mIoU.

The only difference is that I am not using apex and the custom distributed optimizer, so I basically comment:

runner = dict(type='IterBasedRunnerAmp')
fp16 = None
optimizer_config = dict(
    type="DistOptimizerHook",
    update_interval=1,
    grad_clip=None,
    coalesce=True,
    bucket_size_mb=-1,
    use_fp16=True,
)

In the config file.

I run my experiment a single node with 8 GPUs. I was wondering if the performance gap could come from the fact that I am not using DistOptimizerHook and apex, or if there is something else I am missing.

Thanks for your help.

[shallowtoil]

Hi @Adrien987k,

Are you able to reproduce the results without commenting those lines?

[Adrien987k]

Hi @shallowtoil

I was actually not loading the model properly (not interpolating the positional encodings and those were randomly initialised).

Now I am able to recover the performance of the paper so everything is good.