Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AssertionError Boxes #29

Open
mdaehl opened this issue Oct 31, 2023 · 1 comment
Open

AssertionError Boxes #29

mdaehl opened this issue Oct 31, 2023 · 1 comment

Comments

@mdaehl
Copy link

mdaehl commented Oct 31, 2023

Hello, I like your work and wanted to try your code, but I am constantly encountering AssertionErrors (see image). I use the config you recommend for 1 GPU: "--config-file configs/50e.6h.500pro.ignore.yaml --num-gpus 1 SOLVER.IMS_PER_BATCH 2 SOLVER.BASE_LR 0.0025" and crowdhuman data. After debugging I realized that the network predicts nan boxes that cause the error. However, I do not know how to cirumvent this. In your code I found the comment "degenerate boxes gives inf / nan results", but there is no indication how to fix this. Do you have an idea how to solve this problem or how did you handle such boxes? (Note: The loss is decreasing during the training.)

Thanks in advance!

crash
@yexiguafuqihao
Copy link
Collaborator

Try to decrease the learning rate. One case is to replace the learning rate with 0 and observe whether the mistake happens or not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants