Question about the PLC correction

[lauraset]

Hello, pxiangwu. You did a great work in the PLC correction. I am confused when reading the "prob_correction“ function in the utils.py file (line 343-355). You mentioned that " If the predicted confidence exceeds this threshold, we use label correction based on likelihood ratio test. Otherwise, we use probabilistic label correction.“ But the 351 th line "flipped = flipper.multinomial(1, top_probs, 1)[0]" seems to generate a random value. Do the lines 350-355 correspond to the PLC corrrection? By the way, when the prediction has a lower confidence than the confidence threshold, its label must be changed using the codes (350-355) ? I am sorry if I understand wrongly. Thank you for your time.

[pxiangwu]

Hi, actually the prob_correction function contains two strategies: (1) Use the likelihood ratio test (Line 344-348); (2) Use the probabilistic label correction (Line 350-355).

Depending on the confidence level, prob_correction function will automatically choose which strategy to adopt. You can consider this function as a combination of two strategies. Also please note that our PLC framework is general, and any label correction algorithm can be incorporated into it. Thus, here, in the prob_correction function we show two examples of label correction strategies.

Finally, for the probabilistic label correction strategy, it works as follows. For a given sample, we update its label according to the predicted posterior distribution. More specifically, we randomly sample a new label according to this posterior distribution, and then assign it to this sample.

[lauraset]

Hi, pxiangwu. Thanks for your detailed answer. I still have a question about the probabilistic label correction strategy. Will it introduce new noise when adopting the random way based on the predicted posterior distribution? I want to apply this method to semantic segmentaion, so I am doubt this will introduce new noise. Besides, I notice that in the code you released, only in the train_clothing_1M, you chose the prob_correction, while for the datasets (CIFAR and food101), you simply used the lrt_correction. Can we safely replace these settings to prob_correction? Thank you very much.

[pxiangwu]

Hi, for the probabilistic label correction strategy, empirically it achieves better performance in certain situations.

However, please note that, after one iteration, this strategy may introduce new label noise. But it can be proved that this strategy is able to converge and correct the labels eventually (under our iterative label refinement framework). Therefore, prob_correction can also be applied to semantic segmentation. But I am not sure if it would achieve much better performance than the likelihood ratio test for such a task.

In train_clothing_1M, we found that prob_correction achieved better performance than lrt_correction. As a result, we finally chose this strategy for clothing1M. However, for CIFAR and food101, we found that both strategies achieved almost similar performances. Therefore we adopted lrt_correction for these datasets. Also, to demonstrate several examples of our PLC framework, we preserved lrt_correction for CIFAR and food101, and did not replace it with prob_correction in our code.

[lauraset]

Thanks for your reply. I got it.