A quick question

[zhenxianglance]

Dear authors of SPICE,

I am very impressed by your work. Just a quick question: how the semantic prediction matrix P is obtained from the features?

Thanks!

[niuchuangnn]

@zhenxianglance The matrix P is predicted by the CLS head, which takes the features as inputs.

[zhenxianglance]

Thanks for your reply. Here is a follow-up question: how did you initialize the CLS head (e.g. a MLP) to ensure that the accuracy of the top-confident predictions? I guess the accuracy of the pseudo labels is critical to avoid error propagation...

[niuchuangnn]

@zhenxianglance The CLS head is randomly initialized. With optimizing the parameters of the CLS head, the predictions will become better and better, although the initial predictions are not accurate. Because the pseudo-labeling algorithm can correct the predictions to some extent by generating the pseudo labels based on the sample similarity in the embedding space and the predictions in the current iteration. Given the better pseudo labels, the CLS head can be trained for a better prediction, which will further help generate better pseudo labels in the next iteration. This process will be iteratively conducted during training.

The predictions of CLS head are mainly to determine the dissimilarity of different clusters and the embedding features of the representation learning model are to measure the similarity between instance samples.

[zhenxianglance]

Thanks a lot for the details! Very impressive idea!