-
Notifications
You must be signed in to change notification settings - Fork 141
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Performance of DPG Model #19
Comments
Hi,
I always pick an arbitrary stopping epoch or step that is consistent across
all folds.
The reason is that in the 15-fold evaluation setting used in prior papers,
there was no validation set defined. This could not be defined as the final
test performance is on different people and thus a true validation set
should be formed of people neither in the training nor test set. Due to the
size of MPIIGaze (15 people) this isn't feasible.
The configuration in thus repository should include a good stopping
step/epoch already.
Best,
Seonwook
…On Fri, 12 Apr 2019, 06:45 thinkrc, ***@***.***> wrote:
Thanks for your work! I run the dpg_train.py and performed
leave-one-person-out evaluation on MPIIGaze. What is the reasonable way to
count the result, the mean of 15-fold final test results or the mean of
15-fold best live test results?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#19>, or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAuI0ruJ8Lzgj33nUxBSi5Pk7dBxDH2kks5vgA9_gaJpZM4crUci>
.
|
Thanks for your reply! The messages.log shows that the DPG in this repository runs final test at step 052499 across all folds. I calculated the mean angle error of 15 folds at 052499 step and reaches 4.90 degrees on MPIIGaze (not 4.63 degrees). I repeated the experiment three times and got similar results. Could you give some suggestions? |
Ach... I'm very sorry about that. I must have made a mistake in the process of making this available. I can't promise that I can look into this ASAP, but I will try, and keep you updated. |
I went to check my experiment logs and found a discrepancy between my best configuration there, and the version committed to this repo... Please find the changes required for a perf. boost at commit 7087b9e |
Thanks for your work! I run the dpg_train.py and performed leave-one-person-out evaluation on MPIIGaze. What is the reasonable way to count the result, the mean of 15-fold final test results or the mean of 15-fold best live test results?
The text was updated successfully, but these errors were encountered: