fundus dataset source domain split & reproduction result discrepancy

[SHARKALAKALA]

Hi authors,

I am trying to reproduce your outstanding SicTTA work and have a question about the dataset split and experimental results.

In the paper, it is stated that the source domain (Domains A + B) of the fundus dataset was split into an 8:2 ratio for training and validation during source model pre-training. Since the source domain data provided in your GitHub repository does not follow this split ratio, I randomly split the source domain dataset into training and validation sets with an 8:2 ratio by myself for reproduction.However, there is an obvious discrepancy between the reproduction results of the comparison methods and the results in paper: the performance on Domain C is much higher than that reported in the paper, while the performance on Domain D is lower.

Could you please provide the official and specific train/validation split details of the source domain fundus dataset? Are there any other key settings I need to adjust to align with the paper’s experimental setup

[JianghaoWu]

Hi,

Thanks for your interest in our work.

Differences in source-model training can lead to relatively large performance variations. For reference, we have released the source model weights here:
https://github.com/HiLab-git/SicTTA/tree/main/save_model/Fundus-doFE_unet

If you would like to reproduce our reported adaptation results, we recommend using the released source model weights directly.

If you choose to train the source model by yourself, it is normal to observe performance differences.

Best regards