New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tiny Yolo conversion fails to detect any objects #83

Open
seantempesta opened this Issue Nov 6, 2017 · 17 comments

Comments

Projects
None yet
10 participants
@seantempesta
Copy link

seantempesta commented Nov 6, 2017

Converting the darknet tiny-yolo.cfg (only modified for number of classes) fails to detect any objects in Keras.

To be clear:

sean@sean-linux:~/src/darknet/cfg$ diff tiny-yolo.cfg modified-tiny-yolo.cfg 
114c114
< filters=425
---
> filters=30
118c118
< anchors = 0.738768,0.874946,  2.42204,2.65704,  4.30971,7.04493,  10.246,4.59428,  12.6868,11.8741
---
> anchors = 1.82,0.74, 3.09,1.66, 4.04,2.46, 4.87,3.19, 6.32,4.36
120c120
< classes=80
---
> classes=1

Validating the network in Darknet yields very positive results:

./darknet detector recall ./data/modified.data ./cfg/modified-tiny-yolo.cfg ./backup/modified-tiny-yolo_5000.weights -i 0 -thresh 0.2
layer     filters    size              input                output
    0 conv     16  3 x 3 / 1   416 x 416 x   3   ->   416 x 416 x  16
    1 max          2 x 2 / 2   416 x 416 x  16   ->   208 x 208 x  16
    2 conv     32  3 x 3 / 1   208 x 208 x  16   ->   208 x 208 x  32
    3 max          2 x 2 / 2   208 x 208 x  32   ->   104 x 104 x  32
    4 conv     64  3 x 3 / 1   104 x 104 x  32   ->   104 x 104 x  64
    5 max          2 x 2 / 2   104 x 104 x  64   ->    52 x  52 x  64
    6 conv    128  3 x 3 / 1    52 x  52 x  64   ->    52 x  52 x 128
    7 max          2 x 2 / 2    52 x  52 x 128   ->    26 x  26 x 128
    8 conv    256  3 x 3 / 1    26 x  26 x 128   ->    26 x  26 x 256
    9 max          2 x 2 / 2    26 x  26 x 256   ->    13 x  13 x 256
   10 conv    512  3 x 3 / 1    13 x  13 x 256   ->    13 x  13 x 512
   11 max          2 x 2 / 1    13 x  13 x 512   ->    13 x  13 x 512
   12 conv   1024  3 x 3 / 1    13 x  13 x 512   ->    13 x  13 x1024
   13 conv   1024  3 x 3 / 1    13 x  13 x1024   ->    13 x  13 x1024
   14 conv     30  1 x 1 / 1    13 x  13 x1024   ->    13 x  13 x  30
   15 detection
Loading weights from ./backup/modified-tiny-yolo_5000.weights...
 seen 32 
Done!
Learning Rate: 0.001, Momentum: 0.9, Decay: 0.0005
    0     1     1	RPs/Img: 1.00	IOU: 79.94%	Recall:100.00%
    1     2     2	RPs/Img: 1.00	IOU: 79.03%	Recall:100.00%
...
  440   420   441	RPs/Img: 0.99	IOU: 79.02%	Recall:95.24%
  441   421   442	RPs/Img: 0.99	IOU: 79.03%	Recall:95.25%

Conversion to Keras appears to go without any problems:

(yad2kenv) bash-3.2$ python3 yad2k.py -p ../modified-tiny-yolo.cfg ../modified-tiny-yolo_5000.weights model_data/modified-tiny.h5
Using TensorFlow backend.
/Users/sean/src/YOLO-CoreML-MPSNNGraph/Convert/yad2k/yad2kenv/lib/python3.6/importlib/_bootstrap.py:205: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6
  return f(*args, **kwds)
Loading weights.
Weights Header:  [     0      1      0 320000]
Parsing Darknet config.
Creating Keras model.
Parsing section net_0
Parsing section convolutional_0
conv2d bn leaky (3, 3, 3, 16)
2017-11-05 19:40:50.378533: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX
Parsing section maxpool_0
Parsing section convolutional_1
conv2d bn leaky (3, 3, 16, 32)
Parsing section maxpool_1
Parsing section convolutional_2
conv2d bn leaky (3, 3, 32, 64)
Parsing section maxpool_2
Parsing section convolutional_3
conv2d bn leaky (3, 3, 64, 128)
Parsing section maxpool_3
Parsing section convolutional_4
conv2d bn leaky (3, 3, 128, 256)
Parsing section maxpool_4
Parsing section convolutional_5
conv2d bn leaky (3, 3, 256, 512)
Parsing section maxpool_5
Parsing section convolutional_6
conv2d bn leaky (3, 3, 512, 1024)
Parsing section convolutional_7
conv2d bn leaky (3, 3, 1024, 1024)
Parsing section convolutional_8
conv2d    linear (1, 1, 1024, 30)
Parsing section region_0
____________________________________________________________________________________________________
Layer (type)                     Output Shape          Param #     Connected to                     
====================================================================================================
input_1 (InputLayer)             (None, 416, 416, 3)   0                                            
____________________________________________________________________________________________________
convolution2d_1 (Convolution2D)  (None, 416, 416, 16)  432         input_1[0][0]                    
____________________________________________________________________________________________________
batchnormalization_1 (BatchNorma (None, 416, 416, 16)  64          convolution2d_1[0][0]            
____________________________________________________________________________________________________
leakyrelu_1 (LeakyReLU)          (None, 416, 416, 16)  0           batchnormalization_1[0][0]       
____________________________________________________________________________________________________
maxpooling2d_1 (MaxPooling2D)    (None, 208, 208, 16)  0           leakyrelu_1[0][0]                
____________________________________________________________________________________________________
convolution2d_2 (Convolution2D)  (None, 208, 208, 32)  4608        maxpooling2d_1[0][0]             
____________________________________________________________________________________________________
batchnormalization_2 (BatchNorma (None, 208, 208, 32)  128         convolution2d_2[0][0]            
____________________________________________________________________________________________________
leakyrelu_2 (LeakyReLU)          (None, 208, 208, 32)  0           batchnormalization_2[0][0]       
____________________________________________________________________________________________________
maxpooling2d_2 (MaxPooling2D)    (None, 104, 104, 32)  0           leakyrelu_2[0][0]                
____________________________________________________________________________________________________
convolution2d_3 (Convolution2D)  (None, 104, 104, 64)  18432       maxpooling2d_2[0][0]             
____________________________________________________________________________________________________
batchnormalization_3 (BatchNorma (None, 104, 104, 64)  256         convolution2d_3[0][0]            
____________________________________________________________________________________________________
leakyrelu_3 (LeakyReLU)          (None, 104, 104, 64)  0           batchnormalization_3[0][0]       
____________________________________________________________________________________________________
maxpooling2d_3 (MaxPooling2D)    (None, 52, 52, 64)    0           leakyrelu_3[0][0]                
____________________________________________________________________________________________________
convolution2d_4 (Convolution2D)  (None, 52, 52, 128)   73728       maxpooling2d_3[0][0]             
____________________________________________________________________________________________________
batchnormalization_4 (BatchNorma (None, 52, 52, 128)   512         convolution2d_4[0][0]            
____________________________________________________________________________________________________
leakyrelu_4 (LeakyReLU)          (None, 52, 52, 128)   0           batchnormalization_4[0][0]       
____________________________________________________________________________________________________
maxpooling2d_4 (MaxPooling2D)    (None, 26, 26, 128)   0           leakyrelu_4[0][0]                
____________________________________________________________________________________________________
convolution2d_5 (Convolution2D)  (None, 26, 26, 256)   294912      maxpooling2d_4[0][0]             
____________________________________________________________________________________________________
batchnormalization_5 (BatchNorma (None, 26, 26, 256)   1024        convolution2d_5[0][0]            
____________________________________________________________________________________________________
leakyrelu_5 (LeakyReLU)          (None, 26, 26, 256)   0           batchnormalization_5[0][0]       
____________________________________________________________________________________________________
maxpooling2d_5 (MaxPooling2D)    (None, 13, 13, 256)   0           leakyrelu_5[0][0]                
____________________________________________________________________________________________________
convolution2d_6 (Convolution2D)  (None, 13, 13, 512)   1179648     maxpooling2d_5[0][0]             
____________________________________________________________________________________________________
batchnormalization_6 (BatchNorma (None, 13, 13, 512)   2048        convolution2d_6[0][0]            
____________________________________________________________________________________________________
leakyrelu_6 (LeakyReLU)          (None, 13, 13, 512)   0           batchnormalization_6[0][0]       
____________________________________________________________________________________________________
maxpooling2d_6 (MaxPooling2D)    (None, 13, 13, 512)   0           leakyrelu_6[0][0]                
____________________________________________________________________________________________________
convolution2d_7 (Convolution2D)  (None, 13, 13, 1024)  4718592     maxpooling2d_6[0][0]             
____________________________________________________________________________________________________
batchnormalization_7 (BatchNorma (None, 13, 13, 1024)  4096        convolution2d_7[0][0]            
____________________________________________________________________________________________________
leakyrelu_7 (LeakyReLU)          (None, 13, 13, 1024)  0           batchnormalization_7[0][0]       
____________________________________________________________________________________________________
convolution2d_8 (Convolution2D)  (None, 13, 13, 1024)  9437184     leakyrelu_7[0][0]                
____________________________________________________________________________________________________
batchnormalization_8 (BatchNorma (None, 13, 13, 1024)  4096        convolution2d_8[0][0]            
____________________________________________________________________________________________________
leakyrelu_8 (LeakyReLU)          (None, 13, 13, 1024)  0           batchnormalization_8[0][0]       
____________________________________________________________________________________________________
convolution2d_9 (Convolution2D)  (None, 13, 13, 30)    30750       leakyrelu_8[0][0]                
====================================================================================================
Total params: 15,770,510
Trainable params: 15,764,398
Non-trainable params: 6,112
____________________________________________________________________________________________________
None
Saved Keras model to model_data/modified-tiny.h5
Read 15770510 of 15770510.0 from Darknet weights.
Saved model plot to model_data/modified-tiny.png

But no results are detected when testing the keras model (even against the same images that I can detect in Darknet):

(yad2kenv) bash-3.2$  python3 test_yolo.py model_data/modified-tiny.h5 -a model_data/modified-tiny_anchors.txt -c model_data/modified_classes.txt
Using TensorFlow backend.
/Users/sean/src/YOLO-CoreML-MPSNNGraph/Convert/yad2k/yad2kenv/lib/python3.6/importlib/_bootstrap.py:205: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6
  return f(*args, **kwds)
2017-11-05 19:41:17.192237: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX
/Users/sean/src/YOLO-CoreML-MPSNNGraph/Convert/yad2k/yad2kenv/lib/python3.6/site-packages/keras/models.py:150: UserWarning: No training configuration found in save file: the model was *not* compiled. Compile it manually.
  warnings.warn('No training configuration found in save file: '
model_data/modified-tiny.h5 model, anchors, and classes loaded.
Found 0 boxes for 031.JPEG
Found 0 boxes for 032.JPEG
Found 0 boxes for 033.JPEG
Found 0 boxes for dog.jpg
Found 0 boxes for eagle.jpg
Found 0 boxes for giraffe.jpg
Found 0 boxes for horses.jpg
Found 0 boxes for person.jpg
Found 0 boxes for modified_1048.jpg
Found 0 boxes for modified_605.jpg
Found 0 boxes for scream.jpg
@michaeldong

This comment has been minimized.

Copy link

michaeldong commented Jan 4, 2018

I meet same problem

@shady831213

This comment has been minimized.

Copy link

shady831213 commented Jan 14, 2018

same problem …

@shady831213

This comment has been minimized.

Copy link

shady831213 commented Jan 14, 2018

line 79

    yolo_model = load_model(model_path)

to

    yolo_model = load_model(model_path, compile=False)

Fixed

@libennext

This comment has been minimized.

Copy link

libennext commented Jan 17, 2018

@shady831213 Thanks for sharing your solution. Could you tell me which version of tf and keras are you using? I got TypeError: load_model() got an unexpected keyword argument 'compile' when change yolo_model. Thanks! I'm using keras 1.2.2 and tf 1.4.1.

@shady831213

This comment has been minimized.

Copy link

shady831213 commented Jan 17, 2018

@libennext

This comment has been minimized.

Copy link

libennext commented Jan 17, 2018

@shady831213 Thanks! I fixed that TypeError with keras2.1.3, but still got
Found o boxes for xxx.jpg,
anybody has other solution? Thanks!

@shady831213

This comment has been minimized.

Copy link

shady831213 commented Jan 17, 2018

@tgwagner

This comment has been minimized.

Copy link

tgwagner commented Jan 23, 2018

Hi, I have the same issue with 0 boxes found. Has anyone resolved this yet?

@shady831213 Is adding compile=False suppose to fix the issue? I get an error when adding the change to test_yolo.py

File "test_yolo.py", line 79, in _main
yolo_model = load_model(model_path,compile=False)
TypeError: load_model() got an unexpected keyword argument 'compile'

@shady831213

This comment has been minimized.

Copy link

shady831213 commented Jan 24, 2018

This parameter is available in keras2x

@kaka-lin

This comment has been minimized.

Copy link

kaka-lin commented Jan 26, 2018

Hi, I have the same issue with 0 boxes found. Has anyone resolved this yet?

@Willjay90

This comment has been minimized.

Copy link

Willjay90 commented Feb 10, 2018

I also find 0 boxes using tiny-yolo coco cfg and weights

I change weights_file.read(16)) to weights_file.read(20)) in yad2k.py and it works!

 weights_header = np.ndarray(
        shape=(4, ), dtype='int32', buffer=weights_file.read(20))

#104

@MikeShi42 MikeShi42 referenced this issue Apr 14, 2018

Closed

help ? #5

@holger-prause

This comment has been minimized.

Copy link

holger-prause commented May 10, 2018

Thax Weijay - that works - change then mentioned code in yad2k.py does the job. Can you explain what this does and how you got to it?

@Willjay90

This comment has been minimized.

Copy link

Willjay90 commented May 11, 2018

@holger-prause , This is just my opinion, I really don't know for sure.
I just look into the source code in darknet to figure out how to load weights.

In parser.c,

void load_weights(network *net, char *filename) {
    load_weights_upto(net, filename, 0, net->n);
} 

void load_weights_upto(network *net, char *filename, int start, int cutoff) {
  #ifdef GPU
    if(net->gpu_index >= 0){
        cuda_set_device(net->gpu_index);
    }
  #endif
    fprintf(stderr, "Loading weights from %s...", filename);
    fflush(stdout);
    FILE *fp = fopen(filename, "rb");
    if(!fp) file_error(filename);

    int major;
    int minor;
    int revision;
    fread(&major, sizeof(int), 1, fp); // sizeof(int): 4
    fread(&minor, sizeof(int), 1, fp); // sizeof(int): 4
    fread(&revision, sizeof(int), 1, fp); // sizeof(int): 4
    if ((major*10 + minor) >= 2 && major < 1000 && minor < 1000){
        fread(net->seen, sizeof(size_t), 1, fp); // sizeof(size_t): 8
    } else {
        int iseen = 0;
        fread(&iseen, sizeof(int), 1, fp); // sizeof(int): 4
        *net->seen = iseen;
    }
    ...

This is the header part of weight, so I assume it would be 20 bytes (4+4+4+8) in Tiny-YOLO and 16 bytes (4+4+4+4) in YOLO.

@holger-prause

This comment has been minimized.

Copy link

holger-prause commented May 17, 2018

Ah ok - thx for the clarification - even with the source code i am not sure i would have come to the same conclusion - well done!

@oanhrung

This comment has been minimized.

Copy link

oanhrung commented Jun 27, 2018

In class YOLO(object):
function def init(self)
self.model_image_size = (416, 416)
Change (416, 416) to (None, None) --> the result is the same as running with darknet
If your detected boxes is more than 20, add max_boxes in the function call yolo_eval
boxes, scores, classes = yolo_eval(self.yolo_model.output, self.anchors,
len(self.class_names), self.input_image_shape,max_boxes = 100,
score_threshold=self.score, iou_threshold=self.iou)

@kumararvindkumar

This comment has been minimized.

Copy link

kumararvindkumar commented Oct 23, 2018

Hey brother im facing this issue


Layer (type) Output Shape Param # Connected to

input_1 (InputLayer) (None, None, None, 3 0


conv2d_1 (Conv2D) (None, None, None, 3 864 input_1[0][0]


batch_normalization_1 (BatchNor (None, None, None, 3 128 conv2d_1[0][0]


leaky_re_lu_1 (LeakyReLU) (None, None, None, 3 0 batch_normalization_1[0][0]


max_pooling2d_1 (MaxPooling2D) (None, None, None, 3 0 leaky_re_lu_1[0][0]


conv2d_2 (Conv2D) (None, None, None, 6 18432 max_pooling2d_1[0][0]


batch_normalization_2 (BatchNor (None, None, None, 6 256 conv2d_2[0][0]


leaky_re_lu_2 (LeakyReLU) (None, None, None, 6 0 batch_normalization_2[0][0]


max_pooling2d_2 (MaxPooling2D) (None, None, None, 6 0 leaky_re_lu_2[0][0]


conv2d_3 (Conv2D) (None, None, None, 1 73728 max_pooling2d_2[0][0]


batch_normalization_3 (BatchNor (None, None, None, 1 512 conv2d_3[0][0]


leaky_re_lu_3 (LeakyReLU) (None, None, None, 1 0 batch_normalization_3[0][0]


conv2d_4 (Conv2D) (None, None, None, 6 8192 leaky_re_lu_3[0][0]


batch_normalization_4 (BatchNor (None, None, None, 6 256 conv2d_4[0][0]


leaky_re_lu_4 (LeakyReLU) (None, None, None, 6 0 batch_normalization_4[0][0]


conv2d_5 (Conv2D) (None, None, None, 1 73728 leaky_re_lu_4[0][0]


batch_normalization_5 (BatchNor (None, None, None, 1 512 conv2d_5[0][0]


leaky_re_lu_5 (LeakyReLU) (None, None, None, 1 0 batch_normalization_5[0][0]


max_pooling2d_3 (MaxPooling2D) (None, None, None, 1 0 leaky_re_lu_5[0][0]


conv2d_6 (Conv2D) (None, None, None, 2 294912 max_pooling2d_3[0][0]


batch_normalization_6 (BatchNor (None, None, None, 2 1024 conv2d_6[0][0]


leaky_re_lu_6 (LeakyReLU) (None, None, None, 2 0 batch_normalization_6[0][0]


conv2d_7 (Conv2D) (None, None, None, 1 32768 leaky_re_lu_6[0][0]


batch_normalization_7 (BatchNor (None, None, None, 1 512 conv2d_7[0][0]


leaky_re_lu_7 (LeakyReLU) (None, None, None, 1 0 batch_normalization_7[0][0]


conv2d_8 (Conv2D) (None, None, None, 2 294912 leaky_re_lu_7[0][0]


batch_normalization_8 (BatchNor (None, None, None, 2 1024 conv2d_8[0][0]


leaky_re_lu_8 (LeakyReLU) (None, None, None, 2 0 batch_normalization_8[0][0]


max_pooling2d_4 (MaxPooling2D) (None, None, None, 2 0 leaky_re_lu_8[0][0]


conv2d_9 (Conv2D) (None, None, None, 5 1179648 max_pooling2d_4[0][0]


batch_normalization_9 (BatchNor (None, None, None, 5 2048 conv2d_9[0][0]


leaky_re_lu_9 (LeakyReLU) (None, None, None, 5 0 batch_normalization_9[0][0]


conv2d_10 (Conv2D) (None, None, None, 2 131072 leaky_re_lu_9[0][0]


batch_normalization_10 (BatchNo (None, None, None, 2 1024 conv2d_10[0][0]


leaky_re_lu_10 (LeakyReLU) (None, None, None, 2 0 batch_normalization_10[0][0]


conv2d_11 (Conv2D) (None, None, None, 5 1179648 leaky_re_lu_10[0][0]


batch_normalization_11 (BatchNo (None, None, None, 5 2048 conv2d_11[0][0]


leaky_re_lu_11 (LeakyReLU) (None, None, None, 5 0 batch_normalization_11[0][0]


conv2d_12 (Conv2D) (None, None, None, 2 131072 leaky_re_lu_11[0][0]


batch_normalization_12 (BatchNo (None, None, None, 2 1024 conv2d_12[0][0]


leaky_re_lu_12 (LeakyReLU) (None, None, None, 2 0 batch_normalization_12[0][0]


conv2d_13 (Conv2D) (None, None, None, 5 1179648 leaky_re_lu_12[0][0]


batch_normalization_13 (BatchNo (None, None, None, 5 2048 conv2d_13[0][0]

how to fix it??
please help me out.

@kumararvindkumar

This comment has been minimized.

Copy link

kumararvindkumar commented Oct 23, 2018

Im facing here None None None type

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment