warmup training

[nengwp]

When warmup training, the model performance is very poor and the training does not seem to work.

my log:

------------  warmup satrt ----------------
Epoch:0   Accuracy:8.86
Epoch:1   Accuracy:6.71
Epoch:2   Accuracy:6.63
Epoch:3   Accuracy:7.84
Epoch:4   Accuracy:10.38
Epoch:5   Accuracy:10.47
Epoch:6   Accuracy:12.23
Epoch:7   Accuracy:13.41
Epoch:8   Accuracy:12.16
Epoch:9   Accuracy:13.13
------------  warmup end -----------------
Epoch:10   Accuracy:73.77
Epoch:11   Accuracy:75.59
Epoch:12   Accuracy:76.33
Epoch:13   Accuracy:76.40

cifar10 has 10 classes, avg acc = 10

Model performance also tends to be average when training with warmup on my dataset.

In addition, another bug is reported. When idx_unchosen is empty, the loss calculation is nan. Perhaps it should be judged that when idx_unchosen is empty, the pseudo loss is 0.

[Justherozen]

Thank you for pointing this out. We did not observe any anomalies during the warm-up in our experiments (the poor performance may be due to the unturned parameters for different datasets). We will address the issue of idx_unchosen being null in a future fix. We appreciate your feedback very much!