-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
evaluate_angle results #5
Comments
Hi, Thank you for your interest in our work. As I can tell from the plot, you have successfully trained the model. Because our work are not able to handle front back confusion, we only evaluate the samples with angle range from (-90,90) which is demonstrated in the paper. Our evaluation code will map any prediction outside of (-90,90) back to the range. |
As I mentioned above, in our paper and released code, we evaluated samples that have angles within (-90,90). Specifically, in the test set, we filter out the samples that have angles outside of (-90,90) and create a new split for evaluation, i.e., https://github.com/IFICL/SLfM/tree/master/Dataset/AI-Habitat/data-split/hm3d-4view-rotation-filterangle. You can create this split by
|
For the vision model, first of all, I noticed one thing is strange the vision model makes predictions greater than 90 degrees, which doesn't match with my given training config One possible reason it happens is that you don't enable Because we generate the two rotated views with an FOV of 60 degrees, we constrain rotation within 90 degrees to ensure there will be visual signals to pick up |
Got it, I'll try --finer_rotation, thank you so much! |
Hello, I trained the model the slfm_pretext and then slfm_geometric. However, when I evaluate the geometric model, the graph shows that only from 200-1000 are accurately predicted. However, from 0-200 and 1000-1200, are every inaccurately predicted. I was wondering if this is something wrong with the evaluate_angle code. Please look at the attachment
The text was updated successfully, but these errors were encountered: