-
Notifications
You must be signed in to change notification settings - Fork 979
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Direct Covolution Issue #742
Comments
Hi @Alok-Ranjan23, How did you create a convolution? Do you use oneDNN also has a limited support for plain formats provided as-is from user. A user can force convolution to use I ran your convolution using benchdnn, and it seems dispatching fine: $ export DNNL_VERBOSE=1
$ # allow convolution to define memory format for all tensors: src, weights, dst
$ ./tests/benchdnn/benchdnn --conv mb2_ic8oc8_ih222oh222kh9sh1dh0ph4_iw222ow222kw9sw1dw0pw4
...
dnnl_verbose,exec,cpu,convolution,gemm:jit,forward_training,src_f32::blocked:abcd:f0 wei_f32::blocked:abcd:f0 bia_f32::blocked:a:f0 dst_f32::blocked:abcd:f0,,alg:convolution_direct,mb2_ic8oc8_ih222oh222kh9sh1dh0ph4_iw222ow222kw9sw1dw0pw4,155.547
$ # allow convolution to define memory format for weights tensor only.
$ # The src and dst tensors are forced to use NHWC memory format.
$ ./tests/benchdnn/benchdnn --conv --stag=nhwc --dtag=nhwc mb2_ic8oc8_ih222oh222kh9sh1dh0ph4_iw222ow222kw9sw1dw0pw4
...
dnnl_verbose,exec,cpu,convolution,gemm:jit,forward_training,src_f32::blocked:acdb:f0 wei_f32::blocked:cdba:f0 bia_f32::blocked:a:f0 dst_f32::blocked:acdb:f0,,alg:convolution_direct,mb2_ic8oc8_ih222oh222kh9sh1dh0ph4_iw222ow222kw9sw1dw0pw4,156.128
...
$ # force convolution to use fixed memory formats (according to the verbose log)
$ ./tests/benchdnn/benchdnn --conv --stag=nhwc --wtag=abcd --dtag=nhwc mb2_ic8oc8_ih222oh222kh9sh1dh0ph4_iw222ow222kw9sw1dw0pw4
...
dnnl_verbose,exec,cpu,convolution,ref:any,forward_training,src_f32::blocked:acdb:f0 wei_f32::blocked:abcd:f0 bia_f32::blocked:a:f0 dst_f32::blocked:acdb:f0,,alg:convolution_direct,mb2_ic8oc8_ih222oh222kh9sh1dh0ph4_iw222ow222kw9sw1dw0pw4,246.89
... |
No, I have set format_tag as any for weight memory. Please check the attached code `
} int main(int argc, char **argv) {
}` |
Thanks for the reproducer! I put it into a gist gh742.cpp. For me it runs What oneDNN version do you use? Could you please try the latest one? $ ( g++ -std=c++11 gh742.cpp -ldnnl -lpthread && DNNL_VERBOSE=1 ./a.out )
dnnl_verbose,info,oneDNN v1.5.0 (commit f9eae92b4bd12bc96071441ee6361761e371e5b6)
dnnl_verbose,info,cpu,runtime:OpenMP
dnnl_verbose,info,cpu,isa:Intel AVX2
dnnl_verbose,info,gpu,runtime:OpenCL
dnnl_verbose,exec,cpu,reorder,jit:uni,undef,src_f32::blocked:cdba:f0 dst_f32::blocked:abcd:f0,,,8x8x9x9,4.10303
dnnl_verbose,exec,cpu,reorder,jit:uni,undef,src_f32::blocked:acdb:f0 dst_f32::blocked:abcd:f0,,,2x8x222x222,0.868164
dnnl_verbose,exec,cpu,convolution,gemm:jit,forward_inference,src_f32::blocked:abcd:f0 wei_f32::blocked:abcd:f0 bia_f32::blocked:a:f0 dst_f32::blocked:abcd:f0,,alg:convolution_direct,mb2_ic8oc8_ih222oh222kh9sh1dh0ph4_iw222ow222kw9sw1dw0pw4,153.279
dnnl_verbose,exec,cpu,reorder,jit:uni,undef,src_f32::blocked:aBcd8b:f0 dst_f32::blocked:acdb:f0,,,2x8x222x222,0.448975 |
|
I see. The dispatching to gemm-based implementation, which is considered as a fallback, happens because See this restriction: bool ok = ... && jcp.l_pad <= jcp.ur_w // here l_pad = 4 -- width padding,
// ur_w -- register unroll, which is 3
if (!ok) return status::unimplemented; dnnl::impl::cpu::x64::jit_avx2_conv_fwd_kernel_f32::init_conf (jcp=..., cd=..., src_d=..., weights_d=..., dst_d=..., attr=...) at /nfs/pdx/disks/hal9000/emfomenk/ml/fw/ipl_mkl_dnn-master.git.tf/src/cpu/x64/jit_avx2_conv_kernel_f32.cpp:529
529 args_ok = true && jcp.oc % simd_w == 0 && jcp.l_pad <= jcp.ur_w
(gdb) p jcp.l_pad
$1 = 4
(gdb) p jcp.ur_w
$2 = 3 For the original problem (batch=37, ic=104, oc=40, ...) I made a rough comparison of gemm-based and jit-based implementations. To run jit-based implementation I made the padding equal The results are (raw data below):
So, even though gemm-based implementation is slower, at least it is not that horrible -- for the case with ic=oc=8 the difference is much bigger. $ ( OMP_NUM_THREADS=16 ./tests/benchdnn/benchdnn --conv --mode=P -v5 mb37_ic104oc40_ih222kh9sh1dh0ph4 mb37_ic104oc40_ih222kh9sh1dh0ph3 )
run: --conv mb37ic104ih222oc40oh222kh9ph4
oneDNN implementation: gemm:jit
oneDNN implementation: gemm:jit
Output template: perf,%engine%,%name%,%prb%,%Gops%,%Gfreq%,%-time%,%-Gflops%,%0time%,%0Gflops%
perf,cpu,,--conv mb37ic104ih222oc40oh222kh9ph4,1204.42,0,2060.35,584.571,2069.67,581.939
tests:1 passed:0 skipped:0 mistrusted:0 unimplemented:0 failed:0 listed:0
total perf: min(ms):2060.35 avg(ms):2069.67
[emfomenk@jfldnn001 03:35:01 build] (0;0)
$ ( hidebrew; OMP_NUM_THREADS=16 ./tests/benchdnn/benchdnn --conv --mode=P -v5 mb37_ic104oc40_ih222kh9sh1dh0ph3 )
run: --conv mb37ic104ih222oc40oh220kh9ph3
oneDNN implementation: jit:avx2
oneDNN implementation: jit:avx2
Output template: perf,%engine%,%name%,%prb%,%Gops%,%Gfreq%,%-time%,%-Gflops%,%0time%,%0Gflops%
perf,cpu,,--conv mb37ic104ih222oc40oh220kh9ph3,1192.27,0,1192.66,999.679,1214.8,981.455
tests:1 passed:0 skipped:0 mistrusted:0 unimplemented:0 failed:0 listed:0
total perf: min(ms):1192.66 avg(ms):1214.8 |
I am using your C++ API to perform convolution with following dimension
batch = 37,channel = 104, height=222,width=222, no_of_filter = 40,kernel= (9,9),pad_h = 4,pad_w = 4,stride= (1,1).
No_of_filters and channel are multiple of 8 in above case. So i think that C++ api has to take direct convolution path for convolution. But it is taking reference implementation path to perform convolution on these dimension.
DNNL implementation path in ref_convolution_fwd_t::execute_forward [src/cpu/ref_convolution.cpp]
dnnl_primitive_execute cpu,convolution,ref:any,forward_inference,src_f32::blocked:acdb:f0 wei_f32::blocked:abcd:f0 bia_f32::blocked:a:f0 dst_f32::blocked:acdb:f0,,alg:convolution_direct,mb2_ic8oc8_ih222oh222kh9sh1dh0ph4_iw222ow222kw9sw1dw0pw4 54.354
Why are these parameters (in blocked format AVX256 path) not taking direct convolution path instead it is taking reference convolution path to perform convolution?
The text was updated successfully, but these errors were encountered: