Can't run my first agent due to go-vncdriver #167

agarbuno · 2017-03-28T13:13:26Z

Expected behavior

I start python in conda environment after installing gym and universe. I have installed golang and go-vncdriver but I still can't import the universe module. Is there something I am missing?

Actual behavior

Darwin MacBookPro 16.5.0 Darwin Kernel Version 16.5.0: Fri Mar  3 16:52:33 PST 2017; root:xnu-3789.51.2~3/RELEASE_X86_64 x86_64
Python 2.7.13 |Continuum Analytics, Inc.| (default, Dec 20 2016, 23:05:08)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import gym
>>> import universe
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/alfredogarbuno/source-git/universe/universe/__init__.py", line 22, in <module>
    from universe import error, envs
  File "/Users/alfredogarbuno/source-git/universe/universe/envs/__init__.py", line 1, in <module>
    import universe.envs.vnc_env
  File "/Users/alfredogarbuno/source-git/universe/universe/envs/vnc_env.py", line 19, in <module>
    import go_vncdriver
ImportError: dlopen(/Users/alfredogarbuno/source-git/go-vncdriver/go_vncdriver.so, 2): no suitable image found.  Did find:
	/Users/alfredogarbuno/source-git/go-vncdriver/go_vncdriver.so: overlapping segments
	/Users/alfredogarbuno/source-git/go-vncdriver/go_vncdriver.so: overlapping segments

Versions

Please include the result of running

$ uname -a ; python --version; pip show universe gym tensorflow numpy go-vncdriver Pillow

Python 2.7.13 :: Continuum Analytics, Inc.
Name: universe
Version: 0.21.4.dev0
Summary: Universe: a software platform for measuring and training an AI's general intelligence across the world's supply of games, websites and other applications.
Home-page: https://github.com/openai/universe
Author: OpenAI
Author-email: universe@openai.com
License: UNKNOWN
Location: /Users/alfredogarbuno/source-git/universe
Requires: autobahn, docker-py, docker-pycreds, fastzbarlight, go-vncdriver, gym, Pillow, PyYAML, six, twisted, ujson
---
Name: gym
Version: 0.7.4
Summary: The OpenAI Gym: A toolkit for developing and comparing your reinforcement learning agents.
Home-page: https://github.com/openai/gym
Author: OpenAI
Author-email: gym@openai.com
License: UNKNOWN
Location: /Users/alfredogarbuno/anaconda2/envs/unipy/lib/python2.7/site-packages
Requires: numpy, pyglet, requests, six
---
Name: numpy
Version: 1.12.1
Summary: NumPy: array processing for numbers, strings, records, and objects.
Home-page: http://www.numpy.org
Author: NumPy Developers
Author-email: numpy-discussion@scipy.org
License: BSD
Location: /Users/alfredogarbuno/anaconda2/envs/unipy/lib/python2.7/site-packages
Requires:
---
Name: go-vncdriver
Version: 0.4.19
Summary: UNKNOWN
Home-page: UNKNOWN
Author: UNKNOWN
Author-email: UNKNOWN
License: UNKNOWN
Location: /Users/alfredogarbuno/source-git/go-vncdriver
Requires: numpy
---
Name: Pillow
Version: 4.0.0
Summary: Python Imaging Library (Fork)
Home-page: http://python-pillow.org
Author: Alex Clark (Fork Author)
Author-email: aclark@aclark.net
License: Standard PIL License
Location: /Users/alfredogarbuno/anaconda2/envs/unipy/lib/python2.7/site-packages
Requires: olefile

The text was updated successfully, but these errors were encountered:

tarvos21 · 2017-03-28T14:55:24Z

Hi, @agarbuno, I see "no suitable image found" in the error message, so I just guess, maybe it's caused by docker?

agarbuno · 2017-03-28T16:36:33Z

Hi @tarvos21 ! I have never used docker before. So I would not be able to tell if that is the main reason. I know I have it running in my computer. However, following the README.md it doesn't say anything about docker or any way I should be using it for the first agent example. Any help would be really appreciated!

tlbtlbtlb · 2017-03-28T17:33:16Z

This isn't related to docker images. This:

ImportError: dlopen(/Users/alfredogarbuno/source-git/go-vncdriver/go_vncdriver.so, 2): no suitable image found.  Did find:
	/Users/alfredogarbuno/source-git/go-vncdriver/go_vncdriver.so: overlapping segments

could be due to building go_vncdriver for 64-bit, while your Python is 32-bit (or vice-versa).

You didn't mention what OS you're running on. Please include the result of uname -a.

agarbuno · 2017-03-29T09:39:11Z

Hi @tlbtlbtlb , sorry it slipped. Here are the details.

Darwin MacBookPro 16.5.0 Darwin Kernel Version 16.5.0: Fri Mar  3 16:52:33 PST 2017; 
root:xnu-3789.51.2~3/RELEASE_X86_64 x86_64

My Python is 64-bit and the go_vncdriver should also be 64-bit since I build it with

$ git clone https://github.com/openai/go-vncdriver.git
$ cd go-vncdriver
$ python build.py
$ pip install -e .

with my conda environment activated

agarbuno · 2017-03-29T10:18:31Z

Another update, I tried the same with an environment from virtualenv. The output is:

>>> import universe
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "universe/__init__.py", line 22, in <module>
    from universe import error, envs
  File "universe/envs/__init__.py", line 1, in <module>
    import universe.envs.vnc_env
  File "universe/envs/vnc_env.py", line 19, in <module>
    import go_vncdriver
  File "/Users/alfredogarbuno/py2/lib/python2.7/site-packages/go_vncdriver/__init__.py", line 9, in <module>
    from go_vncdriver.go_vncdriver import *
ImportError: dlopen(/Users/alfredogarbuno/py2/lib/python2.7/site-packages/go_vncdriver/go_vncdriver.so, 2): no suitable image found.  Did find:
	/Users/alfredogarbuno/py2/lib/python2.7/site-packages/go_vncdriver/go_vncdriver.so: malformed mach-o image: segment __DWARF vm overlaps segment __LINKEDIT
	/Users/alfredogarbuno/py2/lib/python2.7/site-packages/go_vncdriver/go_vncdriver.so: malformed mach-o image: segment __DWARF vm overlaps segment __LINKEDIT

I google the message malformed mach-o image: segment __DWARF vm overlaps segment __LINKEDIT and it seems its related to Mac OS Sierra. Any other leads in how to fix this?

The information for that environment is very similar to the first one:

Darwin MacBookPro 16.5.0 Darwin Kernel Version 16.5.0: Fri Mar  3 16:52:33 PST 2017; root:xnu-3789.51.2~3/RELEASE_X86_64 x86_64
Python 2.7.10
Name: universe
Version: 0.21.5
Summary: Universe: a software platform for measuring and training an AI's general intelligence across the world's supply of games, websites and other applications.
Home-page: https://github.com/openai/universe
Author: OpenAI
Author-email: universe@openai.com
License: UNKNOWN
Location: /Users/alfredogarbuno/source-git/universe
Requires: autobahn, docker-py, docker-pycreds, fastzbarlight, go-vncdriver, gym, Pillow, PyYAML, six, twisted, ujson
---
Name: gym
Version: 0.8.1
Summary: The OpenAI Gym: A toolkit for developing and comparing your reinforcement learning agents.
Home-page: https://github.com/openai/gym
Author: OpenAI
Author-email: gym@openai.com
License: UNKNOWN
Location: /Users/alfredogarbuno/py2/lib/python2.7/site-packages
Requires: six, requests, pyglet, numpy
---
Name: tensorflow
Version: 1.0.0
Summary: TensorFlow helps the tensors flow
Home-page: http://tensorflow.org/
Author: Google Inc.
Author-email: opensource@google.com
License: Apache 2.0
Location: /Users/alfredogarbuno/py2/lib/python2.7/site-packages
Requires: numpy, six, mock, wheel, protobuf
---
Name: numpy
Version: 1.12.0
Summary: NumPy: array processing for numbers, strings, records, and objects.
Home-page: http://www.numpy.org
Author: NumPy Developers
Author-email: numpy-discussion@scipy.org
License: BSD
Location: /Users/alfredogarbuno/py2/lib/python2.7/site-packages
Requires:
---
Name: go-vncdriver
Version: 0.4.19
Summary: UNKNOWN
Home-page: UNKNOWN
Author: UNKNOWN
Author-email: UNKNOWN
License: UNKNOWN
Location: /Users/alfredogarbuno/py2/lib/python2.7/site-packages
Requires: numpy
---
Name: Pillow
Version: 4.0.0
Summary: Python Imaging Library (Fork)
Home-page: http://python-pillow.org
Author: Alex Clark (Fork Author)
Author-email: aclark@aclark.net
License: Standard PIL License
Location: /Users/alfredogarbuno/py2/lib/python2.7/site-packages
Requires: olefile

gerred · 2017-03-29T23:08:22Z

I'm running into the same issue as well. I have a hunch its this issue: golang/go#19734

gerred · 2017-03-29T23:10:35Z

Just did a quick test. This is being fixed in upstream Go, but for now I'm submitting a PR as a workaround.

agarbuno · 2017-03-29T23:35:28Z

@gerred Perfect! I confirm that this works on OS Sierra with Xcode 8.3. So basically build go-vncdriver following the steps in here. After cloning the repo modify the line (if you're using OpenGL)

def build_gl():
    cmd = 'go build -buildmode=c-shared -o go_vncdriver.so github.com/openai/go-vncdriver'
    eprint('Building with OpenGL: GOPATH={} {}. (Set GO_VNCDRIVER_NOGL to build without OpenGL.)'.format(os.getenv('GOPATH'), cmd))
    return not subprocess.call(cmd.split())

so it reads

def build_gl():
    cmd = 'go build -ldflags -s -buildmode=c-shared -o go_vncdriver.so github.com/openai/go-vncdriver'
    eprint('Building with OpenGL: GOPATH={} {}. (Set GO_VNCDRIVER_NOGL to build without OpenGL.)'.format(os.getenv('GOPATH'), cmd))
    return not subprocess.call(cmd.split())

and the pip-install it!

tlbtlbtlb · 2017-03-30T16:54:23Z

Patch applied to go-vncdriver master: openai/go-vncdriver@d92bf96. Thanks @agarbuno, @gerred, and everyone else who helped track this down.

It was remarkably hard tracking down the problem. I found the solution here: openai/universe#167 Without this fix, I was getting an error that looked like this: ``` > python python_to_go.py Traceback (most recent call last): File "python_to_go.py", line 12, in <module> lib = cdll.LoadLibrary('go_adder/libadder.' + lib_extension()) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py", line 440, in LoadLibrary return self._dlltype(name) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py", line 362, in __init__ self._handle = _dlopen(self._name, mode) OSError: dlopen(go_adder/libadder.dylib, 6): no suitable image found. Did find: go_adder/libadder.dylib: malformed mach-o image: segment __DWARF vm overlaps segment __LINKEDIT /Users/steventroxler/kode/scratch/go-interlang/dyn_langs_to_go/go_adder/libadder.dylib: malformed mach-o image: segment __DWARF vm overlaps segment __LINKEDIT ```

gerred mentioned this issue Mar 29, 2017

Fix building on MacOS Sierra 10.12.4 openai/go-vncdriver#27

Merged

agarbuno closed this as completed Mar 29, 2017

stroxler mentioned this issue Apr 13, 2017

Fix broken dynamic library compile on osx draffensperger/go-interlang#1

Merged

nbro mentioned this issue Sep 18, 2017

ImportError: ... go_vncdriver.so: no suitable image found. Did find: ...go_vncdriver.so: overlapping segments #204

Closed

mirokuratczyk mentioned this issue Aug 14, 2018

Client library fixes Psiphon-Labs/psiphon-tunnel-core#483

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can't run my first agent due to go-vncdriver #167

Can't run my first agent due to go-vncdriver #167

agarbuno commented Mar 28, 2017 •

edited

Loading

tarvos21 commented Mar 28, 2017

agarbuno commented Mar 28, 2017

tlbtlbtlb commented Mar 28, 2017

agarbuno commented Mar 29, 2017 •

edited

Loading

agarbuno commented Mar 29, 2017

gerred commented Mar 29, 2017

gerred commented Mar 29, 2017

agarbuno commented Mar 29, 2017

tlbtlbtlb commented Mar 30, 2017

Can't run my first agent due to go-vncdriver #167

Can't run my first agent due to go-vncdriver #167

Comments

agarbuno commented Mar 28, 2017 • edited Loading

Expected behavior

Actual behavior

Versions

tarvos21 commented Mar 28, 2017

agarbuno commented Mar 28, 2017

tlbtlbtlb commented Mar 28, 2017

agarbuno commented Mar 29, 2017 • edited Loading

agarbuno commented Mar 29, 2017

gerred commented Mar 29, 2017

gerred commented Mar 29, 2017

agarbuno commented Mar 29, 2017

tlbtlbtlb commented Mar 30, 2017

agarbuno commented Mar 28, 2017 •

edited

Loading

agarbuno commented Mar 29, 2017 •

edited

Loading