Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tests not completing successfully #8

Closed
davenso opened this issue Nov 19, 2016 · 5 comments
Closed

Tests not completing successfully #8

davenso opened this issue Nov 19, 2016 · 5 comments
Assignees
@rsdubtso
Labels
bug A confirmed library bug

Comments

@davenso
Copy link

davenso commented Nov 19, 2016 •
edited by vpirogov

System:
Processor: Intel(R) Xeon Phi(TM) CPU 7210 @ 1.30GHz
OS: CentOS Linux release 7.2.1511 (Core)

> [davido@knl-data build]$ make test
> Running tests...
> Test project /home/davido/mkl/mkl-dnn-master/build
>       Start  1: simple-net-c
>  1/23 Test  #1: simple-net-c ..........................***Exception: Other  0.11 sec
>       Start  2: simple-net-cpp
>  2/23 Test  #2: simple-net-cpp ........................***Exception: Other  6.68 sec
>       Start  3: api-c
>  3/23 Test  #3: api-c .................................***Exception: Other  0.25 sec
>       Start  4: test_c_symbols-c
>  4/23 Test  #4: test_c_symbols-c ......................   Passed    0.01 sec
>       Start  5: test_sum
>  5/23 Test  #5: test_sum ..............................   Passed    1.98 sec
>       Start  6: test_reorder
>  6/23 Test  #6: test_reorder ..........................   Passed    0.59 sec
>       Start  7: test_concat
>  7/23 Test  #7: test_concat ...........................   Passed    0.57 sec
>       Start  8: test_relu_forward
>  8/23 Test  #8: test_relu_forward .....................***Failed    0.26 sec
>       Start  9: test_relu_backward
>  9/23 Test  #9: test_relu_backward ....................***Failed    0.25 sec
>       Start 10: test_lrn_forward
> 10/23 Test #10: test_lrn_forward ......................***Failed    1.60 sec
>       Start 11: test_lrn_backward
> 11/23 Test #11: test_lrn_backward .....................***Failed    1.98 sec
>       Start 12: test_pooling_forward
> 12/23 Test #12: test_pooling_forward ..................***Failed    5.49 sec
>       Start 13: test_pooling_backward
> 13/23 Test #13: test_pooling_backward .................   Passed    2.19 sec
>       Start 14: test_batch_normalization_forward
> 14/23 Test #14: test_batch_normalization_forward ......***Failed   10.12 sec
>       Start 15: test_batch_normalization_backward
> 15/23 Test #15: test_batch_normalization_backward .....***Failed   15.45 sec
>       Start 16: test_inner_product_forward
> 16/23 Test #16: test_inner_product_forward ............   Passed    2.47 sec
>       Start 17: test_inner_product_backward_data
> 17/23 Test #17: test_inner_product_backward_data ......   Passed    0.87 sec
>       Start 18: test_inner_product_backward_weights
> 18/23 Test #18: test_inner_product_backward_weights ...   Passed    1.41 sec
>       Start 19: test_convolution_format_any
> 19/23 Test #19: test_convolution_format_any ...........   Passed    0.02 sec
>       Start 20: test_convolution_forward
> 20/23 Test #20: test_convolution_forward ..............***Failed   56.88 sec
>       Start 21: test_convolution_relu_forward
> 21/23 Test #21: test_convolution_relu_forward .........***Failed   40.96 sec
>       Start 22: test_convolution_backward_data
> 22/23 Test #22: test_convolution_backward_data ........   Passed   87.59 sec
>       Start 23: test_convolution_backward_weights
> 23/23 Test #23: test_convolution_backward_weights .....   Passed   90.65 sec
> 
> 48% tests passed, 12 tests failed out of 23
> 
> Total Test time (real) = 328.49 sec
> 
> The following tests FAILED:
>           1 - simple-net-c (OTHER_FAULT)
>           2 - simple-net-cpp (OTHER_FAULT)
>           3 - api-c (OTHER_FAULT)
>           8 - test_relu_forward (Failed)
>           9 - test_relu_backward (Failed)
>          10 - test_lrn_forward (Failed)
>          11 - test_lrn_backward (Failed)
>          12 - test_pooling_forward (Failed)
>          14 - test_batch_normalization_forward (Failed)
>          15 - test_batch_normalization_backward (Failed)
>          20 - test_convolution_forward (Failed)
>          21 - test_convolution_relu_forward (Failed)
> Errors while running CTest
> make: *** [test] Error 8

Dave
@rsdubtso rsdubtso added the bug A confirmed library bug label Nov 20, 2016
@rsdubtso rsdubtso self-assigned this Nov 20, 2016
@rsdubtso
Copy link

Hi Dave,

I am not able to reproduce this with the latest master on my Xeon Phi 7250 system:

$ lsb_release -a
LSB Version:    :core-4.1-amd64:core-4.1-noarch
Distributor ID: RedHatEnterpriseServer
Description:    Red Hat Enterprise Linux Server release 7.2 (Maipo)
Release:        7.2
Codename:       Maipo

$ gcc --version
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-4)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Can you please provide additional details like compiler version and cmake command line? Does your system have selinux enabled?

It would also be great of you could run the simple-net test under gdb and post 'bt' output after an exception.

Thanks,
Roma

@tensor-tang
Copy link
Contributor

tensor-tang commented Nov 23, 2016 •
edited by vpirogov

hi @rsdubtso
I also have the same issue, with Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

Running tests...
Test project /home/tangjian/mkl-dnn-master/build
      Start  1: simple-net-c
 1/23 Test  #1: simple-net-c ..........................***Exception: Other  0.75 sec
      Start  2: simple-net-cpp
 2/23 Test  #2: simple-net-cpp ........................***Exception: Other  0.86 sec
      Start  3: api-c
 3/23 Test  #3: api-c .................................***Exception: Other  0.07 sec
      Start  4: test_c_symbols-c
 4/23 Test  #4: test_c_symbols-c ......................   Passed    0.01 sec
      Start  5: test_sum
 5/23 Test  #5: test_sum ..............................   Passed    0.27 sec
      Start  6: test_reorder
 6/23 Test  #6: test_reorder ..........................   Passed    0.25 sec
      Start  7: test_concat
 7/23 Test  #7: test_concat ...........................   Passed    0.22 sec
      Start  8: test_relu_forward
 8/23 Test  #8: test_relu_forward .....................***Failed    0.13 sec
      Start  9: test_relu_backward
 9/23 Test  #9: test_relu_backward ....................***Failed    0.17 sec
      Start 10: test_lrn_forward
10/23 Test #10: test_lrn_forward ......................***Failed    0.27 sec
      Start 11: test_lrn_backward
11/23 Test #11: test_lrn_backward .....................***Failed    0.24 sec
      Start 12: test_pooling_forward
12/23 Test #12: test_pooling_forward ..................***Failed    0.26 sec
      Start 13: test_pooling_backward
13/23 Test #13: test_pooling_backward .................   Passed    0.22 sec
      Start 14: test_batch_normalization_forward
14/23 Test #14: test_batch_normalization_forward ......***Failed    0.77 sec
      Start 15: test_batch_normalization_backward
15/23 Test #15: test_batch_normalization_backward .....***Failed    0.94 sec
      Start 16: test_inner_product_forward
16/23 Test #16: test_inner_product_forward ............   Passed    0.23 sec
      Start 17: test_inner_product_backward_data
17/23 Test #17: test_inner_product_backward_data ......   Passed    0.22 sec
      Start 18: test_inner_product_backward_weights
18/23 Test #18: test_inner_product_backward_weights ...   Passed    0.22 sec
      Start 19: test_convolution_format_any
19/23 Test #19: test_convolution_format_any ...........   Passed    0.01 sec
      Start 20: test_convolution_forward
20/23 Test #20: test_convolution_forward ..............***Failed    9.25 sec
      Start 21: test_convolution_relu_forward
21/23 Test #21: test_convolution_relu_forward .........***Failed    9.50 sec
      Start 22: test_convolution_backward_data
22/23 Test #22: test_convolution_backward_data ........   Passed   17.64 sec
      Start 23: test_convolution_backward_weights
23/23 Test #23: test_convolution_backward_weights .....   Passed   17.31 sec

48% tests passed, 12 tests failed out of 23

Total Test time (real) =  59.83 sec

The following tests FAILED:
          1 - simple-net-c (OTHER_FAULT)
          2 - simple-net-cpp (OTHER_FAULT)
          3 - api-c (OTHER_FAULT)
          8 - test_relu_forward (Failed)
          9 - test_relu_backward (Failed)
         10 - test_lrn_forward (Failed)
         11 - test_lrn_backward (Failed)
         12 - test_pooling_forward (Failed)
         14 - test_batch_normalization_forward (Failed)
         15 - test_batch_normalization_backward (Failed)
         20 - test_convolution_forward (Failed)
         21 - test_convolution_relu_forward (Failed)
Errors while running CTest
make: *** [test] Error 8

Could u help?

$ gcc --version
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-4)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

@jczaja
Copy link
Contributor

jczaja commented Nov 23, 2016

Not sure if this relevant, but I can see some of code using MKLDNN failing when selinux is set into enrofcing mode, while it works fine when permissive/disabled mode is set. So please attach output from:
"sestatus" command.

@tensor-tang
Copy link
Contributor

tensor-tang commented Nov 23, 2016 •
edited

Yes, it's enabled.
What wonderful it works now when permissive is set. I have tried so many ways which does not work.
Thanks very much! @jczaja

@taliesinb
Copy link

taliesinb commented Nov 23, 2016 •
edited

@tensor-tang you might want to format your tool output as markdown code, because github actually interprets all those #n entries as being references to issue pages. So an issue I'm watching got pinged by your tool output.

@Darwinian2 Darwinian2 mentioned this issue Aug 9, 2017
Closed
@xyzsam xyzsam mentioned this issue Dec 13, 2017
Closed
@greenpdx greenpdx mentioned this issue Dec 30, 2017
Closed
@mrinmayk mrinmayk mentioned this issue Jan 9, 2018
Closed
@moting9 moting9 mentioned this issue Apr 16, 2018
Closed
@ghost ghost mentioned this issue Sep 21, 2018
Closed
@angus1121 angus1121 mentioned this issue Jan 9, 2020
Closed
@kakulo kakulo mentioned this issue Jun 11, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rsdubtso rsdubtso

Labels
bug A confirmed library bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@taliesinb @davenso @jczaja @rsdubtso @tensor-tang