-
Notifications
You must be signed in to change notification settings - Fork 7.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How does 'route' layer work in yolov2? #487
Comments
Route-layer is the same as concat-lyaer in the Caffe. More: #120 (comment) |
In layer-25, you mean we take the result of layer-16 as output of layer-25 as well as input of layer-26. But how do we deal with the output of layer-24? If we drop it, layer-17-24 will be meaningless. |
So the route-layer and reorg-layer are a module to concatenate current output (like layer-24's output) and previous output(like layer-16's output), which means 'bring finer grained features in from earlier in the network'. Here's another question I'm wondering. My object detection is slow. I use yolo-voc.cfg network 416x416 on your fork and get 9.2s/ one picture on average. And my CPU is eight i7-7700K Core with 39.76 GFlops/computer[1]. Due to some restrictions, I cannot use GPU. According to this issue #80, I should achieve about ~0.01 FPS per 1 GFlops-SP. So I should have gotten 0.3976 FPS and ~2.5s/ one picture. Could you figure out what's the problem? Thanks a lot! |
~0.01 FPS per 1 GFlops-SP - only if all CPU-resources are used. But Darknet Yolo well optimized only for GPU, but not for CPU. I.e. Darknet doesn't use SSE3/4/AVX (SIMD) optimizations, so it slower about 3-4x times than it could be. I added Yolo v2 to the OpenCV: opencv/opencv#9705 So if you want to use Yolo on CPU then the fastest way is to use Yolo v2 that built-in OpenCV since 3.4.0 - it can process ~2.5s/ one picture and faster:
|
Thanks for your advice! And I tried your first solution -- set |
Yes, So it actually works much faster. |
Get it! You really help me a lot! I just need to use an another timer function! |
@AlexeyAB Do you mean you build two versions of Yolo v2? One is built without OpenCV, the other one is built in OpenCV 3.4, and the latter one is faster? Can the latter one support OpenCV 3.1? Thanks~ |
I mean that OpenCV 3.4.0 already contains Yolo v2 for CPU inside OpenCV. So you can just install OpenCV 3.4.0 without installation Darknet - and you can use the fastest version of Yolo v2 for CPU.
Yolo v2 that built-in OpenCV only since 3.4.0. But this repository you can use with any OpenCV version. |
|
If
And
|
@AlexeyAB |
@AlexeyAB I can download dnn.hpp individually to update the file, but it may confuse some people. Could you try to figure it out? Thanks. |
@wsyzzz
If you want to use my repo, just switch to the branch |
If forward is y = x + F(x) + sqrt(ReLU(x)*ReLU(F(x)) + 0.01) https://github.com/tiffany0107/SORT-Layer/blob/master/sort_layer.cpp says for 2 inputs x1 / x2: |
@ralek67 @AlexeyAB |
If ReLU is leaky: In Tiffany SORT Layer it is = 0 Forward : y = x + ??? top_diff = dy it's delta for backpropagation In my formula it is the inverse: it should be dx1/dy = 1 + ... Hope you understand |
@ralek67 Could you share the process of getting your formula of the gradient? For forward, if y = x + F(x) + sqrt(ReLU(x)*ReLU(F(x)) + 0.001 and ReLU is leaky, SORT_short_cut can be implemented by replacing float sqrt_shift=0.001;
out[out_index] = out[out_index] + add[add_index] + sqrtf(max(sqrt_shift,out[out_index] * add[add_index]+sqrt_shift)); Is it correct? the understand of the following code is correct?
|
I don't know how to add it in Darknet tbh. and it should be: But according to the author of paper, it was implemented like this: If you just read Backward_cpu function for instance, you get to: And that's exactly what math says if ReLU isn't leaky: Thing is you keep talking about y = x + F(x) instead of 2 inputs y = x1 + x2 But that's just 16 years old student maths. |
(1) So, the forward pass y = x + F(x) + sqrt( x*F(x) + sqrtshift ) , sqrtshift = 0.001and ReLU is linear for the shorcut layer in YOLOv3, If out denotes the input value x=state.net.layers[l.index].output, and add denotes F(x)=state.input which is the output value of the last layer, then SORT_short_cut can be implemented with float sqrt_shift=0.001;
out[out_index] = out[out_index] + add[add_index] + sqrt(max(0.0,out[out_index]) * max(0.0,add[add_index])+sqrt_shift); in the funciton shortcut_cpu() of blas.c @ralek67 Thank you very much! Although I have spent three days to understand functions of the shortcut layer, I still do not kown how to implemente the backward of ! (3) Meanwhile, the forward [shortcut] is y=x + F(x) in YOLOv3, then how to understand the code of the backward i.e, backward_shortcut_layer()?
(4)Then how to understand axpy_cpu() and shortcut_cpu() in backward of original shortcut layer according to the processe of gradient computation shown in the above diagram? void backward_shortcut_layer(const layer l, network_state state)
{
gradient_array(l.output, l.outputs*l.batch, l.activation, l.delta);
axpy_cpu(l.outputs*l.batch, 1, l.delta, 1, state.delta, 1);
shortcut_cpu(l.batch, l.out_w, l.out_h, l.out_c, l.delta, l.w, l.h, l.c, state.net.layers[l.index].delta); //dx=dx+dy
} |
Hi everyone, does anyone know how 'route' layer work in yolov2? I google it and only find that "the route layer is to bring finer grained features in from earlier in the network". So how it 'bring finer grained features in'?
E.g. I provide a case to explain. I use
./darknet detector test cfg/coco.data cfg/yolo.cfg yolo.weights data/dog.jpg
And in 'yolo.cfg', I found that
So It's clear that the 25th route layer uses 16th layer for 'layers=-9' in .cfg file, the 28th route layer uses 27th and 24th layer for 'layers=-1, -3' in .cfg file.
Here is my question. How the route layer how the 25th route layer uses 13×13×1024 input from the 24th layer and 26×26×512 input from the 16th layer to obtain 26×26×512 output(according to the 26th input)?
I am looking for any advice. Thanks for your response!
The text was updated successfully, but these errors were encountered: