Model training performance does not improve.

[emma000730]

Hello! I downloaded your preprocessed Synapse dataset and trained according to the instructions, but the training results did not improve. What could be the reason for this?

[McGregorWwww]

Hi, due to class-imbalance, the training of the Synapse dataset is highly unstable when using early-stopping (influenced by randomness), maybe you can change the validation interval to a smaller number, e.g., 5 or 2 or even 1 in

DHC/code/train_dhc.py

Line 364 in 9756413

if epoch_num % 10 == 0:

, or enlarge the early_stop_patience.

[emma000730]

Is this data too low? The best evaluation dice is 0.13395972549915314.
I modified the validation interval, but it seems it didn't help improve performance

[McGregorWwww]

Yes, it should be around 0.4, you can check our training logs in the weights downloading link.

[McGregorWwww]

Maybe you need to check whether the data is correctly processed, e.g., through visualization.

[emma000730]

Thank you for your response. I have two more questions to ask: 1. The results of running another AMOS dataset are similar to yours, but the Dice score for the Synapse dataset is still around 0.15. Why is there such a difference when I move your dataset and code to my computer? Is this related to the small number of samples in the Synapse dataset? Would you give me some advice? 2. Why does the loss during training appear negative, even as large as -9 or -10?

[McGregorWwww]

1. It is indeed due to the small number of samples in the Synapse dataset, there are several parameters you can modify: learning rate, batchsize, larger weight of unsupervised loss, smaller accumulate_iters of DiffDW and smaller momentum of DistDW.

2. We apply -dice as the loss rather than 1-dice, so the loss can be negative; when weighted with the class weights, it can be smaller than -1.