Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

the reason why 'no detections found by model'? #628

Closed
2hyes opened this issue Feb 8, 2021 · 14 comments
Closed

the reason why 'no detections found by model'? #628

2hyes opened this issue Feb 8, 2021 · 14 comments

Comments

@2hyes
Copy link

2hyes commented Feb 8, 2021

I want to do transfer learning with using Darknet53.conv.74 backbone, coco128 datasets.

---- mAP 0.0
Training Epoch 1: 100%|█████████████████████████| 16/16 [00:10<00:00,  1.59it/s]
Training Epoch 2: 100%|█████████████████████████| 16/16 [00:09<00:00,  1.61it/s]

---- Evaluating Model ----
Detecting objects: 100%|████████████████████████| 16/16 [00:02<00:00,  5.77it/s]
metrics_output:  None
---- mAP not measured (no detections found by model)

This is the result of $ python train.py --data_config data/coco.data --pretrained_weights weights/darknet53.conv.74.

Here, why does detection not work? (I didn't change code.)

@Flova
Copy link
Collaborator

Flova commented Feb 8, 2021

You need more than 2 epochs to train the network on coco. Even if you are using some pre-trained weights in the encoder.

@2hyes
Copy link
Author

2hyes commented Feb 9, 2021

스크린샷 2021-02-09 오전 10 25 33

yes I used 10epochs, and every detection task (except right after 0epoch) does not work. (refer to image)

/data/yolov3/coco128/images/train2017/000000000009.jpg
/data/yolov3/coco128/images/train2017/000000000025.jpg
/data/yolov3/coco128/images/train2017/000000000030.jpg
/data/yolov3/coco128/images/train2017/000000000034.jpg
/data/yolov3/coco128/images/train2017/000000000036.jpg
...

and my train.txt and valid.txt file is like this.

@Flova
Copy link
Collaborator

Flova commented Feb 9, 2021

That is indeed weird. Maybe I give it a try in the next days.

@2hyes
Copy link
Author

2hyes commented Feb 9, 2021

I think because of dataset. I will try training with larger dataset, and let you know soon in here.
Thank you.

@Flova
Copy link
Collaborator

Flova commented Feb 9, 2021

But it should easily overfit on this smaller set. Maybe there is something wrong with the formatting and the import.

@Zzheng-6
Copy link

Did you solve this problem? Because I also encountered this error

@Flova
Copy link
Collaborator

Flova commented Feb 10, 2021

On the same dataset?

@2hyes
Copy link
Author

2hyes commented Feb 14, 2021

@Zzheng-6 not yet.

I tried transfer learning with the following conditions.

  • dataset: 500 sampled images of COCO2014 dataset
  • pretrained weights = darknet53.conv.74
  • evaluation_interval = 2
| 66    | keyboard       | 0.00000 |
| 67    | cell phone     | 0.00000 |
| 68    | microwave      | 0.00000 |
| 69    | oven           | 0.00000 |
| 70    | toaster        | 0.00000 |
| 71    | sink           | 0.00001 |
| 72    | refrigerator   | 0.00007 |
| 73    | book           | 0.00002 |
| 74    | clock          | 0.00000 |
| 75    | vase           | 0.00000 |
| 76    | scissors       | 0.00000 |
| 77    | teddy bear     | 0.00003 |
| 79    | toothbrush     | 0.00000 |
+-------+----------------+---------+
---- mAP 2.601784971674012e-05

Training Epoch 1: 100%|██████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.62it/s]
---- training loss 341.9720458984375

Training Epoch 2: 100%|██████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.51it/s]
---- training loss 304.5881652832031


---- Evaluating Model ----
Detecting objects:   0%|                                                                        | 0/63 [00:00<?, ?it/s]

Result is like this.
First detecting task is working, but the second detecting task is not working and just disconnects the server I use.

@Zzheng-6
Copy link

Zzheng-6 commented Feb 14, 2021 via email

@Dan944
Copy link

Dan944 commented Mar 13, 2021

I met the same problem, it seems that all the conf are too low, which makes that the map is quite low even the test set are train set

@cesarhcq
Copy link

@Zzheng-6 not yet.

I tried transfer learning with the following conditions.

  • dataset: 500 sampled images of COCO2014 dataset
  • pretrained weights = darknet53.conv.74
  • evaluation_interval = 2
| 66    | keyboard       | 0.00000 |
| 67    | cell phone     | 0.00000 |
| 68    | microwave      | 0.00000 |
| 69    | oven           | 0.00000 |
| 70    | toaster        | 0.00000 |
| 71    | sink           | 0.00001 |
| 72    | refrigerator   | 0.00007 |
| 73    | book           | 0.00002 |
| 74    | clock          | 0.00000 |
| 75    | vase           | 0.00000 |
| 76    | scissors       | 0.00000 |
| 77    | teddy bear     | 0.00003 |
| 79    | toothbrush     | 0.00000 |
+-------+----------------+---------+
---- mAP 2.601784971674012e-05

Training Epoch 1: 100%|██████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.62it/s]
---- training loss 341.9720458984375

Training Epoch 2: 100%|██████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.51it/s]
---- training loss 304.5881652832031


---- Evaluating Model ----
Detecting objects:   0%|                                                                        | 0/63 [00:00<?, ?it/s]

Result is like this.
First detecting task is working, but the second detecting task is not working and just disconnects the server I use.

What is the image size which are you using?

@2hyes
Copy link
Author

2hyes commented Mar 17, 2021

I use 'image_size = 416'.

@Flova
Copy link
Collaborator

Flova commented Apr 11, 2021

I tried to train it with the coco128 dataset, but it takes significantly longer than 10 Epochs to make useful detections. Because of the low object probability for a given cell, it's best for the network to predict nothing in the beginning. I also updated the loss function. Maybe try again.

@Flova
Copy link
Collaborator

Flova commented Apr 11, 2021

@Zzheng-6 not yet.

I tried transfer learning with the following conditions.

* dataset: 500 sampled images of COCO2014 dataset

* pretrained weights = darknet53.conv.74

* evaluation_interval = 2
| 66    | keyboard       | 0.00000 |
| 67    | cell phone     | 0.00000 |
| 68    | microwave      | 0.00000 |
| 69    | oven           | 0.00000 |
| 70    | toaster        | 0.00000 |
| 71    | sink           | 0.00001 |
| 72    | refrigerator   | 0.00007 |
| 73    | book           | 0.00002 |
| 74    | clock          | 0.00000 |
| 75    | vase           | 0.00000 |
| 76    | scissors       | 0.00000 |
| 77    | teddy bear     | 0.00003 |
| 79    | toothbrush     | 0.00000 |
+-------+----------------+---------+
---- mAP 2.601784971674012e-05

Training Epoch 1: 100%|██████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.62it/s]
---- training loss 341.9720458984375

Training Epoch 2: 100%|██████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.51it/s]
---- training loss 304.5881652832031


---- Evaluating Model ----
Detecting objects:   0%|                                                                        | 0/63 [00:00<?, ?it/s]

Result is like this.
First detecting task is working, but the second detecting task is not working and just disconnects the server I use.

This looks like an issue due to the evaluation. During training, no non-maximum suppression is applied. Evaluation on the other hand uses nms. The nms scales not well for large quantities of predicted objects. So if the network predicts many wrong objects in the beginning it is able to get "stuck" in the evaluation. I lowered the timeout for the nms estimation because a wrong nms doesn't matter that much in a situation where we have unrealistic amounts of objects.

@Flova Flova closed this as completed Jun 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants