Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue about sh file parameters to reproduce the results of the TADPOLE challenge in the paper #1

Closed
CUI-J opened this issue May 11, 2022 · 2 comments

Comments

@CUI-J
Copy link

CUI-J commented May 11, 2022

Hello, I am very interested in your work and started trying to reproduce this work. I'm validating based on the TADPOLE challenge data first, but based on your script, the training and testing results are much lower than what is reported in your paper. In addition, I see in your Readme file that the configuration based on that sh file can reproduce the results in the paper, but I tried it and the performance was poor. Therefore, I would like to ask if there are other parameters that need to be set? Looking forward to your reply, thanks!

@SsGood
Copy link
Owner

SsGood commented May 11, 2022

I'm so sorry!
The data preprocessing causes the problem, I can't believe I forgot to standardize the data when I reorganized the codes.
Now I upload a new .ipynb file for data preprocessing of TADPOLE, which fixes the bug. Now you could run the code on the data after standardization, and get a normal result.
But since I cannot guarantee the step of data_standardization after the reorganization is as same as the original code, I will upload a new .sh file to reproduce the result a few days later.

Thank you for your help in making us find this bug!

@CUI-J
Copy link
Author

CUI-J commented May 12, 2022

Thank you for your reply.
After you modified the data processing code, the results of the experiment have been greatly improved. Based on the current .sh file configuration, I tried to run the TADPOLE three-category experiment. The test acc and auc are 91.47% and 92.42% respectively, which are very close to the results reported in the paper!

Very grateful for your help!

@CUI-J CUI-J closed this as completed May 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants