-
Notifications
You must be signed in to change notification settings - Fork 572
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to train yolov4-csp via darknet ? #13
Comments
|
@WongKinYiu Thanks ! |
@WongKinYiu yolov4-csp where? |
I dont think you can use yolov4.conv.137 in this case, since there are many differences between these networks. You either need to train from scratch or create your own conv file from csp weights |
@kadirbeytorun how to create conv file from weights file ? Thanks |
I trained for my own custom dataset via darknet , the mAP always is zero and avg loss from 1000 to 2000
|
There is some serious problem with your dataset or cfg file. You need share more information about your dataset and also share your cfg file here if you want to get help |
@kadirbeytorun
|
I dont see anything weird with the cfg file. Did you perhaps create your conv file wrong? Maybe you couldn't create it properly, so you cannot do transfer learning, and your network is trying to learn it from scratch Show me how you created your conv file. |
@kadirbeytorun |
Did you try to train with Or did you try to train with |
@AlexeyAB ,OK , I will test it and let you know . |
@AlexeyAB can I can set new_coords=0 and opitmized_memeory=0 at the same time to train ? |
@AlexeyAB Thanks ,I change new_coords=0, The training of my custom dataset works ok ! but the avg loss still very big training log as below:
|
@toplinuxsir I did some fixes. Try to download the latest Darknet version. And use for each [yolo] layer
|
@AlexeyAB OK, I will test it again , and Let you know. |
the file size: 162M, Thanks |
All these files are in the assets at the bottom: https://github.com/AlexeyAB/darknet/releases/tag/darknet_yolo_v4_pre
You can use any cfg-file. |
@AlexeyAB
the darknent and cfg file, conv file all download form github the latest darkenet version
but the training is not normal ,avg loss is very big and ap is zero , the log as below
|
@AlexeyAB The training iterations is mAp is always 0, |
|
@AlexeyAB Ok, I will test with the latest darknet version and let you know the result . |
With previous darknet version:
I will test with the latest darknet version and let you know the result . |
Just be sure that you can see objects on your image after resizing it to 640x640 resolution. |
Yes I can see objects after resizeing to 640X640 , The dataset training works ok for yolov4. |
with the latest darknet version, trained for 1167 interations , the mAP@0.5 is still zero
the mAP is still zero. below as the training log:
|
@AlexeyAB I use the latest darknet version trained for more than 6500 interations ,the mAP still zero |
I trained use the latest commit AlexeyAB/darknet@8d6e56e , Tensor Cores are disabled until the first 3000 iterations are reached. calculation mAP (mean average precision)... for conf_thresh = 0.25, precision = -nan, recall = 0.00, F1-score = -nan IoU threshold = 50 %, used Area-Under-Curve for each unique Recall Set -points flag: mean_average_precision (mAP@0.5) = 0.000000
|
yolov4-csp.cfg (320x320) b=32 on MS COCO: |
@AlexeyAB I use you latest commit AlexeyAB/darknet@4709f61 , with new_coords=1, avg loss is nan after 1000 interations |
|
my training command
yolov4x-mish.cfg
|
I trained for more than 9000 iterations but the mAP is still zero ,I will use you latest commit AlexeyAB/darknet@c47b24a train again . |
Try to use the latest commit with new_coord=1. And use logistic activation instead of linear before [yolo] layers:
mAP increases faster with new_coord=1 than with new_coord=0 |
@AlexeyAB your new commit seems ok |
@AlexeyAB I trained for more than 6000 interations , I got the mAP for my custom dataset is: 99.45% , avg loss is 845(very big), but the opencv dnn model doese not work with yolov4x-mish. |
@AlexeyAB The newest darknet version dose not save training results for every 1000 interations, Only two files : the best and the last. |
@toplinuxsir if you are using Alexey darknet it saves every 10000 iterations, to change it to 1000 iteration go to src folder/ detector.c open it, line 385 you will found if the condition changes every 10000 with 1000 and it will work, I try it and it works for me. if ((iteration >= (iter_save + 1000) || iteration % 1000 == 0) || |
@abdulghani91 |
@toplinuxsir You mean it will save weight every 1000 iteration but for me every time I have to change the condition to make it save for every 1000 iteration, the latest version saves weights for every 10000itr. |
@abdulghani91 Are you sure using the latest commit |
@toplinuxsir I'm using google colab to train, and I try it before only for every 10000itr will store a weight file (10000, 20000, 30000), I try that before two weeks, but after I did the change on line 385 in the detector.c file the darknet start storing every 1000it, and I don't know if the condition that I change it right I just change every 10000 to 1000, and I clone the darknet from this link (https://github.com/AlexeyAB/darknet) |
when I open the detector.c file in the Alexey darknet I found this condition: but for the fixed that you mentioned they change the condition to: |
@toplinuxsir is this (AlexeyAB/darknet@b5ff7f4) the latest version or there is a newer one. |
Hi @toplinuxsir and @WongKinYiu, I was working on YOLOV4 with the latest version of Alexey Repo but I got the problem of -NAN loss. here is the output of my training log When I train YOLOV4 the training log goes from iteration number 2 with the following output where some of the numbers are not -nan 2: -nan, -nan avg loss, 0.000000 rate, 34.954638 seconds, 128 images, 32.050234 hours left to iteration number 147 where all the numbers become -nan 147: -nan, -nan avg loss, 0.000000 rate, 23.773124 seconds, 9408 images, 20.681359 hours left any idea what the solution will be? |
Hi @toplinuxsir and @AlexeyAB , I am trying to fix same problem. class: 12 Thanks. |
Hi @AlexeyAB, if the image resolution I use for training in Yolov4 CSP is different, should I enable "random=1" ? |
I would like to use yolov4-leaky cfg file. If so, which weight file should I use? |
@AlexeyAB [yolo] now, I get alot of bounding boxes!!! |
Tensor Cores are disabled until the first 3000 iterations are reached. cuDNN Error: CUDNN_STATUS_BAD_PARAM after iteration 999, the training process suddenly stop, even though the number of max_batches=6000 |
How to train yolov4-csp using darknet ?
Thanks
The text was updated successfully, but these errors were encountered: