Questions about NormedLinear_Classifier

[JingLiJJ]

Thanks for your great work. I am trying to use NormedLinear_Classifier since I may modify it later. But, it is not good now. Could you please tell me the hyperparameters when your training, such as lr, supt in loss?

[jiequancui]

Hi,

Thanks for your question.

The hyperparameters in our experiments can be found in our training scripts (https://github.com/dvlab-research/Parametric-Contrastive-Learning/tree/main/LT/sh or https://github.com/dvlab-research/Parametric-Contrastive-Learning/tree/main/Full-ImageNet/sh).

We don't use the normed classifier in our work because we observed that a normed classifier is easier to overfit for the classification task when the model is fully trained though it may have some advantages with 90 epochs or 180 epochs training.

If you need to adopt a normed classifier, you may need to set a proper temperature for similarities by learnable centers and features (see https://github.com/dvlab-research/Parametric-Contrastive-Learning/blob/main/LT/losses.py #41).

I'm very glad to discuss more on the problem if needed.

[JingLiJJ]

Hi,

Thanks for your question.

The hyperparameters in our experiments can be found in our training scripts (https://github.com/dvlab-research/Parametric-Contrastive-Learning/tree/main/LT/sh or https://github.com/dvlab-research/Parametric-Contrastive-Learning/tree/main/Full-ImageNet/sh).

We don't use the normed classifier in our work because we observed that a normed classifier is easier to overfit for the classification task when the model is fully trained though it may have some advantages with 90 epochs or 180 epochs training.

If you need to adopt a normed classifier, you may need to set a proper temperature for similarities by learnable centers and features (see https://github.com/dvlab-research/Parametric-Contrastive-Learning/blob/main/LT/losses.py #41).

I'm very glad to discuss more on the problem if needed.

Thanks for your quick reply!! Based on your experience, it would be better to set the temperatures of centers and features to the same or different?

[jiequancui]

Hi,

I'm not quite sure. You may need to conduct some experiments to verify it.
Maybe you can fix self.temperature = 0.2 and then try some different values for self.supt.

Like in LDAM, it usually needs a scale of around 30 for the normed classifier in classification. (https://github.com/kaidic/LDAM-DRW/blob/master/losses.py #45)

[JingLiJJ]

Thanks!!!