Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test results of official trained model(download_PieAPPv0.1_PT_weights.sh) not same with paper #13

Open
JiadiMo opened this issue May 28, 2022 · 2 comments

Comments

@JiadiMo
Copy link

JiadiMo commented May 28, 2022

Hi, I have tested the trained model(download_PieAPPv0.1_PT_weights.sh) that you released, but the performance is lower than the result in your paper by about 2% in both PLCC, SRCC, and KRCC. Also, the reimplement is also lower than the result in your paper by about 2%. Can I ask for your testing and training details?

Thanks a lot.

@Luciennnnnnn
Copy link

@JiadiMo Hi, I also wondering how test their models, can you share your experiences?

@ch-andrei
Copy link

ch-andrei commented Dec 26, 2023

@JiadiMo What SROCC did you achieve?

FYI, on my end, after a quick test with the provided pretrained weights and similar (although modified) setup
SROCC=0.79185
KROCC=0.60356
PLCC=0.78609
RMSE=0.09523
on PieAPP test set.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants