Validation loss show a dot on evaluation graphs

[Annieliaquat]

I am using SSD MobileNet V2 320x320 for training. When I am run eval.py it only shows dot in the middle of each graph on Tensor board rather than a complete graph. This is the issue for both validation loss and mAP
Please guide. Does anyone know how to solve it? thanks

[laxmareddyp]

Hi @Annieliaquat,
In order to expedite the trouble-shooting process, please provide a colab/code snippet to reproduce the issue reported here. Thanks!

[Annieliaquat]

Hi @Annieliaquat, In order to expedite the trouble-shooting process, please provide a colab/code snippet to reproduce the issue reported here. Thanks!

Here is the code I use to generate evaluation metrics for my model.

!python DetectionApi/models/research/object_detection/model_main_tf2.py --model_dir=DetectionApi/workspace/models/eval-ssd-fpnlite --pipeline_config_path=DetectionApi/workspace/models/ssd-fpnlite-320/pipeline.config --checkpoint_dir=/content/drive/MyDrive/DetectionApi/workspace/models/ssd-fpnlite-320/

[laxmareddyp]

Hi @Annieliaquat,

The evaluation script has a default timeout of 3600 seconds to wait for a new checkpoint to be generated as the script was initially intended to be running in parallel to the training job, but we are running it after the training process in Colab. So it will take the latest checkpoint and do evaluation on eval dataset provided and only display a single dot for mAP in evaluation graph of tensor board.

Thanks.

[google-ml-butler]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.