Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

clarification evaluation #31

Closed
F-Barto opened this issue Apr 14, 2022 · 1 comment
Closed

clarification evaluation #31

F-Barto opened this issue Apr 14, 2022 · 1 comment

Comments

@F-Barto
Copy link

F-Barto commented Apr 14, 2022

Hello and many thanks for your work and sharing your code.

I have a question regarding the way you compute your IoU metric and how it compares against Lift-splat.

You use stat_scores_multiple_classes from PLmetrics to compute the iou. Correct me if I am wrong, but by default the threshold of this method is 0.5

On the other hand, in get_batch_iou of LFS they use a threshold of 0: pred = (preds > 0)
https://github.com/nv-tlabs/lift-splat-shoot/blob/master/src/tools.py

Wouldn't this have an impact on the evaluation results ,and thus, on how you compare to them ?
@F-Barto
Copy link
Author

F-Barto commented Apr 14, 2022 •
edited
Loading

My bad, LFS uses preds > 0 where preds is before sigmoid, which makes it > 0.5 after sigmoid

@F-Barto F-Barto closed this as completed Apr 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@F-Barto