Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The log does not contain the evaluated value. Can the epoch be made smaller? Which indicator represents Overall performance? #21

Open
mrsempress opened this issue Apr 11, 2024 · 5 comments

Comments

@mrsempress
Copy link

  1. The log file only contains loss information, and the evaluation information is only printed in the terminal, which is very unfriendly for epoch 400 training. I have to test each epoch again.
  2. Is epoch 400 necessary? In my experiments, val loss became overfitting near epoch 50. Can the epoch be adjusted to 75 or other relatively small values? Will it affect the results? Have you done any corresponding experiments?
  3. How long does it take to run the complete 400 epochs?
  4. In your public training log, it only reaches epoch 72. In issue 3, I find you load epoch 60, and the overall result is from last_semantic alignment Acc0.25: Top-1: 0.54586.
    But in the training log, the results of epoch 60 are:
last_ Box given span (soft-token) Acc0.50: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
last_ Box given span (contrastive) Acc0.25: Top-1: 0.546, Top-5: 0.680, Top-10: 0.736
last_ Box given span (contrastive) Acc0.50: Top-1: 0.423, Top-5: 0.573, Top-10: 0.627
proposal_ Box given span (soft-token) Acc0.25: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
proposal_ Box given span (soft-token) Acc0.50: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
proposal_ Box given span (contrastive) Acc0.25: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
proposal_ Box given span (contrastive) Acc0.50: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
0head_ Box given span (soft-token) Acc0.25: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
0head_ Box given span (soft-token) Acc0.50: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
0head_ Box given span (contrastive) Acc0.25: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
0head_ Box given span (contrastive) Acc0.50: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
1head_ Box given span (soft-token) Acc0.25: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
1head_ Box given span (soft-token) Acc0.50: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
1head_ Box given span (contrastive) Acc0.25: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
1head_ Box given span (contrastive) Acc0.50: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
2head_ Box given span (soft-token) Acc0.25: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
2head_ Box given span (soft-token) Acc0.50: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
2head_ Box given span (contrastive) Acc0.25: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
2head_ Box given span (contrastive) Acc0.50: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
3head_ Box given span (soft-token) Acc0.25: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
3head_ Box given span (soft-token) Acc0.50: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
3head_ Box given span (contrastive) Acc0.25: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
3head_ Box given span (contrastive) Acc0.50: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
4head_ Box given span (soft-token) Acc0.25: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
4head_ Box given span (soft-token) Acc0.50: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
4head_ Box given span (contrastive) Acc0.25: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000
4head_ Box given span (contrastive) Acc0.50: Top-1: 0.000, Top-5: 0.000, Top-10: 0.000

Why is one segment and one box?

@mrsempress
Copy link
Author

  1. In addition, I would like to know the specific meanings of last_ Box and proposal_ Box, soft-token and contrastive.
  2. I want to know which values should correspond to the Acc@0.25(det) and Acc@0.5(det) results of Unique (∼19%) and Multiple (∼81%).

@mrsempress
Copy link
Author

For question 6, is it from Analysis Unique and multi?

@mrsempress
Copy link
Author

  1. it reports last_ position and last_ semantic in validation. What are their respective meanings?

@mrsempress
Copy link
Author

  1. Under what circumstances is it appropriate to stop training, or in other words, the optimal performance is generally around the epoch number

@mrsempress
Copy link
Author

mrsempress commented Apr 11, 2024

  1. How long is the testing time? It takes me 45 minutes here. But in your log is 11'20. I use the command srun -p test --gres=gpu:1 sh test_scanrefer.sh. What did I do wrong?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant