Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ops test core dump #47

Open
BingWin789 opened this issue Jan 30, 2020 · 5 comments
Open

ops test core dump #47

BingWin789 opened this issue Jan 30, 2020 · 5 comments

Comments

@BingWin789
Copy link

Hi:
I compiled the hdrnet ops under python 3.5.2 (virtualenv) tensorflow 1.8.0 CUDA 10 cuDNN 7.1.4 ubuntu 16.04. Then I test ops by test_ops.py. The first two tests are passed. But the third test 'test_grid_gradient' encountered a 'core dump' error. The detail infomation is as follows:

`test/ops_test.py Fatal Python error: Aborted

Thread 0x00007fde827c0700 (most recent call first):
File "/data/mabing/PROGRAMFILES/tf18/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1409 in _call_tf_sessionrun
File "/data/mabing/PROGRAMFILES/tf18/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1307 in _run_fn
File "/data/mabing/PROGRAMFILES/tf18/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1322 in _do_call
File "/data/mabing/PROGRAMFILES/tf18/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1316 in _do_run
File "/data/mabing/PROGRAMFILES/tf18/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1135 in _run
File "/data/mabing/PROGRAMFILES/tf18/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 900 in run
File "/data/mabing/PROGRAMFILES/tf18/lib/python3.5/site-packages/tensorflow/python/ops/gradient_checker.py", line 115 in _compute_theoretical_jacobian
File "/data/mabing/PROGRAMFILES/tf18/lib/python3.5/site-packages/tensorflow/python/ops/gradient_checker.py", line 237 in _compute_gradient
File "/data/mabing/PROGRAMFILES/tf18/lib/python3.5/site-packages/tensorflow/python/ops/gradient_checker.py", line 326 in compute_gradient
File "/data/mabing/PROGRAMFILES/tf18/lib/python3.5/site-packages/tensorflow/python/ops/gradient_checker.py", line 371 in compute_gradient_error
File "/data/mabing/CODES/HDRnets/tstops/test/ops_test.py", line 127 in test_grid_gradient
File "/usr/lib/python3.5/unittest/case.py", line 600 in run
File "/usr/lib/python3.5/unittest/case.py", line 648 in call`

I want to know if I should modify source code of bilateral slice to adapt to tf1.8 or just I tested it in a wrong way. Thanks in advance.

@mgharbi
Copy link
Owner

mgharbi commented Feb 13, 2020

I have never tested it on 1.8, so it is possible this version introduced breaking C++ API changes. Unfortunately, I will not have time to look into this any time soon.

@mygmyg
Copy link

mygmyg commented Feb 19, 2020

Hi, what is the version of tf and cuda that you have tested? I make it to adapt tf2.1 ,but the gradient is not right.Thanks.

@eduardinjo
Copy link

@mygmyg Do you mean that tf2.1 produces incorrect results? I upgraded to tf1.14, also most of the tests gave 'core dump' errors but it still converges to reasonable output although I haven't checked if all values are correct.

@YinengXiong
Copy link

@eduardinjo I encountered
Check failed: work_element_count > 0 (-1072000000 vs. 0)
Aborted (core dumped)
when I run ops_test.py, did you resolve this problem?

@tzhong518
Copy link

tzhong518 commented Jun 20, 2020

I'm using 1.13. Not sure if is the same problem but I also met FATAL ERROR when using py.test to run the test.

test/ops_test.py FFatal Python error: Aborted

Current thread 0x00002b82c4da9580 (most recent call first):
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 676 in __init__
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1551 in __init__
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/tensorflow/python/framework/test_util.py", line 2454 in _create_session
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/tensorflow/python/framework/test_util.py", line 2464 in _get_cached_session
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/tensorflow/python/framework/test_util.py", line 1697 in test_session
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/contextlib.py", line 81 in __enter__
  File "/mnt/datagrid1/tzhong/hdrnet/hdrnet/test/ops_test.py", line 222 in test_grid_optimize
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/unittest/case.py", line 605 in run
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/unittest/case.py", line 653 in __call__
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/_pytest/unittest.py", line 231 in runtest
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/_pytest/runner.py", line 135 in pytest_runtest_call
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/_pytest/runner.py", line 217 in <lambda>
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/_pytest/runner.py", line 244 in from_call
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/_pytest/runner.py", line 217 in call_runtest_hook
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/_pytest/runner.py", line 186 in call_and_report
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/_pytest/runner.py", line 100 in runtestprotocol
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/_pytest/runner.py", line 85 in pytest_runtest_protocol
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/_pytest/main.py", line 272 in pytest_runtestloop
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/_pytest/main.py", line 247 in _main
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/_pytest/main.py", line 191 in wrap_session
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/_pytest/main.py", line 240 in pytest_cmdline_main
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/pluggy/manager.py", line 87 in <lambda>
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/lib/python3.6/site-packages/_pytest/config/__init__.py", line 125 in main
  File "/home/tzhong/.pyenv/versions/anaconda3-5.3.0/envs/tf1_13/bin/py.test", line 8 in <module>

I added if name == 'main' func in ops_test and used python to run it, which seemed to work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants