Fedora 24 unable to import sonnet #23

gabrielleyr · 2017-04-20T15:28:39Z

Fedora 24, tensorflow 1.0.1, bazel 0.4.5
Installing sonnet seems to have been successful (Requirement already satisfied: sonnet==1.0.....), including installing jdk8, bazel, sonnet, and the ./configure for tensorflow. But when I try to import sonnet, I get the error below. Any suggestions?

import sonnet as snt
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python2.7/site-packages/sonnet/init.py", line 102, in
from sonnet.python.ops.resampler import resampler
File "/usr/lib/python2.7/site-packages/sonnet/python/ops/resampler.py", line 33, in
tf.resource_loader.get_path_to_datafile("_resampler.so"))
File "/usr/lib/python2.7/site-packages/tensorflow/python/framework/load_library.py", line 64, in load_op_library
None, None, error_msg, error_code)
tensorflow.python.framework.errors_impl.NotFoundError: /usr/lib/python2.7/site-packages/sonnet/python/ops/_resampler.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringB5cxx11Ev

However, works if I switch to the sonnet directory, then import, but then testing it I get an ImportError:

import sonnet as snt
import tensorflow as tf
snt.resampler(tf.constant([0.]), tf.constant([0.]))
Traceback (most recent call last):
File "", line 1, in
File "sonnet/python/ops/resampler.py", line 65, in resampler
raise ImportError("_gen_resampler could not be imported.")
ImportError: _gen_resampler could not be imported.

I did uninstall sonnet before installing the whl file.

malcolmreynolds · 2017-04-20T16:05:14Z

Hi,

The second error is expected, as you're trying to import from the source directory, which doesn't have the compiled library available.

As for the first, seems like some symbols that Sonnet expects to be present are not. Can you run "nm $file | grep CheckOpMessageBuilder" on the sonnet _resampler.so file, as well as $SITE_PACKAGES_DIR/tensorflow/python/_pywrap_tensorflow.so.

For reference, I see the following (on Mac):

$ nm /tmp/sonnet/sonnet/python/ops/_resampler.so | grep CheckOpMessageBuilder
0000000000001a30 t __ZN10tensorflow8internal21CheckOpMessageBuilder7ForVar1Ev
                 U __ZN10tensorflow8internal21CheckOpMessageBuilder7ForVar2Ev
                 U __ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEv
                 U __ZN10tensorflow8internal21CheckOpMessageBuilderC1EPKc
                 U __ZN10tensorflow8internal21CheckOpMessageBuilderD1Ev

$ nm _pywrap_tensorflow.so  | grep CheckOpMessageBuilder
0000000001b8de00 T __ZN10tensorflow8internal21CheckOpMessageBuilder7ForVar2Ev
0000000001b8de30 T __ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEv
0000000001b8ddb0 T __ZN10tensorflow8internal21CheckOpMessageBuilderC1EPKc
0000000001b8dc80 T __ZN10tensorflow8internal21CheckOpMessageBuilderC2EPKc
0000000001b8dde0 T __ZN10tensorflow8internal21CheckOpMessageBuilderD1Ev
0000000001b8ddc0 T __ZN10tensorflow8internal21CheckOpMessageBuilderD2Ev

ie, the resampler file contains references to a bunch of external symbols, but they are all defined in the main tensorflow static library. I suspect you will see that you don't have those symbols available, which should help us debug further.

gabrielleyr · 2017-04-20T17:59:20Z

$ nm /usr/lib/python2.7/site-packages/sonnet/python/ops/_resampler.so | grep CheckOpMessageBuilder
00000000000176ba W _ZN10tensorflow8internal21CheckOpMessageBuilder7ForVar1Ev
U _ZN10tensorflow8internal21CheckOpMessageBuilder7ForVar2Ev
U _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringB5cxx11Ev
U _ZN10tensorflow8internal21CheckOpMessageBuilderC1EPKc
U _ZN10tensorflow8internal21CheckOpMessageBuilderD1Ev

As for $SITE_PACKAGES_DIR/tensorflow/python/_pywrap_tensorflow.so, the output is a huge list of similarly formatted lines.

malcolmreynolds · 2017-04-20T22:34:56Z

Can you post the full output of nm $SITE_PACKAGES_DIR/tensorflow/python/_pywrap_tensorflow.so | grep CheckOpMessageBuilder ?

sathishreddy · 2017-04-21T02:41:34Z

Hi,
I am also facing the simialr issue; My enviroment is Ubuntu 16.04, Tensorflow '1.0.1'. I am getting follwoing error, when I try to import sonnet

----> 1 import sonnet as snt
/home/sathish/anaconda3/envs/py27/lib/python2.7/site-packages/sonnet/__init__.py in <module>()
    100 from sonnet.python.ops import nest
    101 from sonnet.python.ops.initializers import restore_initializer
--> 102 from sonnet.python.ops.resampler import resampler
    103 from sonnet.python.ops.resampler import resampler_is_available

/home/sathish/anaconda3/envs/py27/lib/python2.7/site-packages/sonnet/python/ops/resampler.py in <module>()
     31   # Link the shared object.
     32   _resampler_so = tf.load_op_library(
---> 33       tf.resource_loader.get_path_to_datafile("_resampler.so"))
     34 
     35 

/home/sathish/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/framework/load_library.pyc in load_op_library(library_filename)
     62       # pylint: disable=protected-access
     63       raise errors_impl._make_specific_exception(
---> 64           None, None, error_msg, error_code)
     65       # pylint: enable=protected-access
     66   finally:

NotFoundError: /home/sathish/anaconda3/envs/py27/lib/python2.7/site-packages/sonnet/python/ops/_resampler.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringB5cxx11Ev

This is output from the command nm _resampler.so | grep CheckOpMessageBuilder
0000000000017bfa W _ZN10tensorflow8internal21CheckOpMessageBuilder7ForVar1Ev U _ZN10tensorflow8internal21CheckOpMessageBuilder7ForVar2Ev U _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringB5cxx11Ev U _ZN10tensorflow8internal21CheckOpMessageBuilderC1EPKc U _ZN10tensorflow8internal21CheckOpMessageBuilderD1Ev

This is the output form command: nm _pywrap_tensorflow.so | grep CheckOpMessageBuilder

00000000023ba280 T _ZN10tensorflow8internal21CheckOpMessageBuilder7ForVar2Ev 00000000023ba2b0 T _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEv 00000000023ba0a0 T _ZN10tensorflow8internal21CheckOpMessageBuilderC1EPKc 00000000023ba0a0 T _ZN10tensorflow8internal21CheckOpMessageBuilderC2EPKc 00000000023ba250 T _ZN10tensorflow8internal21CheckOpMessageBuilderD1Ev 00000000023ba250 T _ZN10tensorflow8internal21CheckOpMessageBuilderD2Ev

floodsung · 2017-04-21T04:37:13Z

i have the same error. my environment is Ubuntu16.04. I installed sonnet in virtualenv

floodsung · 2017-04-21T04:38:44Z

floodsung@floodsung:~/Documents/sonnet$ bazel build --config=opt :install
Warning: ignoring http_proxy in environment.
Extracting Bazel installation...
.........
WARNING: Config values are not defined in any .rc file: opt
WARNING: /home/floodsung/.cache/bazel/_bazel_floodsung/7e8a8460c09d6f45cdda8156f9433431/external/org_tensorflow/tensorflow/workspace.bzl:72:5: tf_repo_name was specified to tf_workspace but is no longer used and will be removed in the future.
WARNING: /home/floodsung/Documents/sonnet/sonnet/python/BUILD:119:1: in srcs attribute of cc_binary rule //sonnet/python:ops/_resampler.so: please do not import '//sonnet/cc:ops/resampler.cc' directly. You should either move the file to this package or depend on an appropriate rule there. Since this rule was created by the macro 'tf_custom_op_library', the error might have been caused by the macro implementation in /home/floodsung/Documents/sonnet/sonnet/tensorflow.bzl:108:25.
WARNING: /home/floodsung/Documents/sonnet/sonnet/python/BUILD:119:1: in srcs attribute of cc_binary rule //sonnet/python:ops/_resampler.so: please do not import '//sonnet/cc/kernels:resampler_op.cc' directly. You should either move the file to this package or depend on an appropriate rule there. Since this rule was created by the macro 'tf_custom_op_library', the error might have been caused by the macro implementation in /home/floodsung/Documents/sonnet/sonnet/tensorflow.bzl:108:25.
WARNING: /home/floodsung/Documents/sonnet/sonnet/python/BUILD:119:1: in srcs attribute of cc_binary rule //sonnet/python:ops/_resampler.so: please do not import '//sonnet/cc/kernels:resampler_op.h' directly. You should either move the file to this package or depend on an appropriate rule there. Since this rule was created by the macro 'tf_custom_op_library', the error might have been caused by the macro implementation in /home/floodsung/Documents/sonnet/sonnet/tensorflow.bzl:108:25.

Do you have the same WARNINGs?

sathishreddy · 2017-04-21T04:40:44Z

@songrotek : yeah

sathishreddy · 2017-04-21T06:04:28Z

I followed the steps described by @roman3017 in issue number #5. Now sonnet is working. Seems to be some lib refences issues between tensorflow 1.0.1 and sonnet:
Please use the below instructions to install sonnet:

cd sonnet/tensorflow
git checkout r1.1
./configure
bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
sudo -H pip uninstall tensorflow
sudo -H pip install /tmp/tensorflow_pkg/tensorflow-1.1.0rc1-cp27-cp27mu-linux_x86_64.whl
cd ../
bazel build --config=opt //:install
bazel-bin/install /tmp/sonnet_pkg
sudo -H pip uninstall sonnet
sudo -H pip install /tmp/sonnet_pkg/sonnet-1.0-py2-none-any.whl

avrono · 2017-04-24T15:44:30Z

I am having the same problem on Linux Mint 18.1. I followed the proceedure above and the output I get is

 import sonnet as snt
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/home/avrono/TensorFlow/local/lib/python2.7/site-packages/sonnet/__init__.py", line 102, in 
<module>
        from sonnet.python.ops.resampler import resampler
  File "/home/avrono/TensorFlow/local/lib/python2.7/site-packages/sonnet/python/ops/resampler.py", line 33, in <module>
    tf.resource_loader.get_path_to_datafile("_resampler.so"))
  File "/home/avrono/TensorFlow/local/lib/python2.7/site-
packages/tensorflow/python/framework/load_library.py", line 64, in load_op_library
   None, None, error_msg, error_code)
tensorflow.python.framework.errors_impl.NotFoundError: 
/home/avrono/TensorFlow/local/lib/python2.7/site-packages/sonnet/python/ops/_resampler.so: 
undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringB5cxx11Ev

gabrielleyr · 2017-04-24T17:56:06Z

I also followed the procedure suggested by @sathishreddy for installing TF1.1 (just leaving out --config=cuda). Within the sonnet dir I had the same '_gen_resampler could not be imported' error as before, but outside it was able to successfully run the resampler test. Thanks so much!

curtismwatson · 2017-05-09T15:28:59Z

I used pip to upgrade TensorFlow to 1.1 but having an error when trying to configure the headers to bazel build:

[cmwatson@xx tensorflow]$ ./configure
Please specify the location of python. [Default is /usr/bin/python]:
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]:
Do you wish to use jemalloc as the malloc implementation? [Y/n]
jemalloc enabled
Do you wish to build TensorFlow with Google Cloud Platform support? [y/N]
No Google Cloud Platform support will be enabled for TensorFlow
Do you wish to build TensorFlow with Hadoop File System support? [y/N]
No Hadoop File System support will be enabled for TensorFlow
Do you wish to build TensorFlow with the XLA just-in-time compiler (experimental)? [y/N]
No XLA support will be enabled for TensorFlow
Found possible Python library paths:
  /usr/lib/python2.7/site-packages
  /usr/lib64/python2.7/site-packages
Please input the desired Python library path to use.  Default is [/usr/lib/python2.7/site-packages]

Using python library path: /usr/lib/python2.7/site-packages
Do you wish to build TensorFlow with OpenCL support? [y/N]
No OpenCL support will be enabled for TensorFlow
Do you wish to build TensorFlow with CUDA support? [y/N]
No CUDA support will be enabled for TensorFlow
Configuration finished
INFO: Starting clean (this may take a while). Consider using --expunge_async if the clean takes more than several minutes.
.........
ERROR: package contains errors: tensorflow.
ERROR: no such package '@io_bazel_rules_closure//closure': error loading package 'external': Could not load //external package.

I am able to import tensorflow, so is there an option that I'm missing on 'configure'?

curtismwatson · 2017-05-09T19:35:43Z

Previous issue was from old bazel version. I upgraded to 0.4.5 -- now having this issue which I cannot make sense.


[cmwatson@xx tensorflow]$ ./configure
Please specify the location of python. [Default is /usr/bin/python]:
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]:
Do you wish to use jemalloc as the malloc implementation? [Y/n]
jemalloc enabled
Do you wish to build TensorFlow with Google Cloud Platform support? [y/N]
No Google Cloud Platform support will be enabled for TensorFlow
Do you wish to build TensorFlow with Hadoop File System support? [y/N]
No Hadoop File System support will be enabled for TensorFlow
Do you wish to build TensorFlow with the XLA just-in-time compiler (experimental)? [y/N]
No XLA support will be enabled for TensorFlow
Found possible Python library paths:
  /usr/lib/python2.7/site-packages
  /usr/lib64/python2.7/site-packages
Please input the desired Python library path to use.  Default is [/usr/lib/python2.7/site-packages]

Using python library path: /usr/lib/python2.7/site-packages
Do you wish to build TensorFlow with OpenCL support? [y/N]
No OpenCL support will be enabled for TensorFlow
Do you wish to build TensorFlow with CUDA support? [y/N]
No CUDA support will be enabled for TensorFlow
Configuration finished
Warning: ignoring http_proxy in environment.
INFO: Starting clean (this may take a while). Consider using --expunge_async if the clean takes more than several minutes.
Warning: ignoring http_proxy in environment.
...........
ERROR: /home/cmwatson/sonnet/tensorflow/tensorflow/tensorboard/bower/BUILD:5:1: no such package '@weblas_weblas_js//file': Error downloading [https://raw.githubusercontent.com/waylonflinn/weblas/v0.9.0/dist/weblas.js] to /home/cmwatson/.cache/bazel/_bazel_cmwatson/69b7f2b22f6b880ca0a532b8cb646acd/external/weblas_weblas_js/weblas.js: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target and referenced by '//tensorflow/tensorboard/bower:bower'.
ERROR: /home/cmwatson/.cache/bazel/_bazel_cmwatson/69b7f2b22f6b880ca0a532b8cb646acd/external/com_google_dagger/BUILD:13:1: no such package '@com_google_dagger_compiler//': java.io.IOException: Error downloading [http://domain-registry-maven.storage.googleapis.com/repo1.maven.org/maven2/com/google/dagger/dagger-compiler/2.8/dagger-compiler-2.8.jar, http://maven.ibiblio.org/maven2/com/google/dagger/dagger-compiler/2.8/dagger-compiler-2.8.jar, http://repo1.maven.org/maven2/com/google/dagger/dagger-compiler/2.8/dagger-compiler-2.8.jar] to /home/cmwatson/.cache/bazel/_bazel_cmwatson/69b7f2b22f6b880ca0a532b8cb646acd/external/com_google_dagger_compiler/dagger-compiler-2.8.jar: Tried to reconnect at offset 8,481,153 but server didn't support it and referenced by '@com_google_dagger//:com_google_dagger'.
ERROR: Evaluation of query "deps(((//tensorflow/... - //tensorflow/contrib/nccl/...) - //tensorflow/examples/android/...))" failed: errors were encountered while computing transitive closure.

gabrielleyr closed this as completed Apr 24, 2017

sathishreddy mentioned this issue Apr 26, 2017

ipython train.py -- --memory_size=64 --num_bits=8 --max_length=3 NotFoundError google-deepmind/dnc#4

Closed

curtismwatson mentioned this issue May 9, 2017

Fedora 24 Sonnet Installation Issue #31

Closed

This was referenced May 21, 2017

fixed! #32

Closed

Mac install fails #29

Closed

Bazel build failure on Ubuntu14.04 #37

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fedora 24 unable to import sonnet #23

Fedora 24 unable to import sonnet #23

gabrielleyr commented Apr 20, 2017 •

edited

malcolmreynolds commented Apr 20, 2017 •

edited

gabrielleyr commented Apr 20, 2017

malcolmreynolds commented Apr 20, 2017

sathishreddy commented Apr 21, 2017

floodsung commented Apr 21, 2017

floodsung commented Apr 21, 2017

sathishreddy commented Apr 21, 2017

sathishreddy commented Apr 21, 2017 •

edited

avrono commented Apr 24, 2017

gabrielleyr commented Apr 24, 2017

curtismwatson commented May 9, 2017

curtismwatson commented May 9, 2017

Fedora 24 unable to import sonnet #23

Fedora 24 unable to import sonnet #23

Comments

gabrielleyr commented Apr 20, 2017 • edited

malcolmreynolds commented Apr 20, 2017 • edited

gabrielleyr commented Apr 20, 2017

malcolmreynolds commented Apr 20, 2017

sathishreddy commented Apr 21, 2017

floodsung commented Apr 21, 2017

floodsung commented Apr 21, 2017

sathishreddy commented Apr 21, 2017

sathishreddy commented Apr 21, 2017 • edited

avrono commented Apr 24, 2017

gabrielleyr commented Apr 24, 2017

curtismwatson commented May 9, 2017

curtismwatson commented May 9, 2017

gabrielleyr commented Apr 20, 2017 •

edited

malcolmreynolds commented Apr 20, 2017 •

edited

sathishreddy commented Apr 21, 2017 •

edited