Failed when import py3d on Windows 10 #408

HaiDang2001VN · 2018-06-22T15:25:59Z

Hello @syncle , thank you and your team with magnificent works in computer vision and graphic processing.
I have compiled Open3D with specified Python 3.5 and VS 2017 (CMake use std:c++14 when compile). Then I built INSTALL and it successfully created py3d.cp35-win_amd64.pyd on C:\Users\...\Python\Python 3.5\site-packages\, but when I use Python IDLE 3.5 (64-bit) to import py3d, it get no module named error.
I checked the system variables and make new var named PYTHONPATH (as it doesn't exist), assigned the path of .pyd to it but that seems not getting better.
I also found the same .pyd in build\lib\Python\Release and run python interpreter to import it, received ImportError again.
Did you know about anything caused this error or what info you need more to understand the problem here?

Info:
OS: Windows 10 64-bit (Ver 1803, Build 17134.112)
VS 2017: MSVC 14.4 and CMake auto-find OpenMP, Python
Python 3.5 (64-bit) and dependencies is along with Open3D (src\External folder)
Thanks in advance.

The text was updated successfully, but these errors were encountered:

takanokage · 2018-06-22T15:33:21Z

@HaiDang2001VN
when was the last time you pulled down the Open3D code?
we have changed the py3d name to open3d quite a while ago.

please checkout the latest version and try again. alternatively you can try

pip install open3d-python

HaiDang2001VN · 2018-06-22T15:52:33Z

Hello @takanokage ,
I cloned the repo 3 hours ago and then built it.
Before cloned it, I eventually try:
pip3 install open3d-python
then import open3d from Python interpreter but it produced
ImportError: DLL load failed: The specified module could not be found.
I don't know what DLL it used so I decied to directly compile from source, but it results as above.
So, alternatively if you know what DLL it needs, please tell me.
Thanks in advance.

takanokage · 2018-06-22T17:33:08Z

do you have more details on the error?
what version of open3d-python did you try? the latest I uploaded is 0.1.5.5, the previous ones where displaying indeed the error you're describing.

I just tested again and it works on my Windows machine with Python 2.7 and 3.5, 32bit and 64bit.

qianyizh · 2018-06-22T17:35:57Z

I have a similar issue on OSX.
I pip installed, then get:

>>> import open3d
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/qianyizh/anaconda/envs/py36/lib/python3.6/site-packages/open3d/__init__.py", line 11, in <module>
    from open3d.macos import *
  File "/Users/qianyizh/anaconda/envs/py36/lib/python3.6/site-packages/open3d/macos/__init__.py", line 7, in <module>
    globals().update(importlib.import_module('open3d.macos.open3d').__dict__)
  File "/Users/qianyizh/anaconda/envs/py36/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: dlopen(/Users/qianyizh/anaconda/envs/py36/lib/python3.6/site-packages/open3d/macos/open3d.so, 2): Library not loaded: /usr/local/opt/glew/lib/libGLEW.2.1.dylib
  Referenced from: /Users/qianyizh/anaconda/envs/py36/lib/python3.6/site-packages/open3d/macos/open3d.so
  Reason: image not found

Does it mean I need to install glew?

HaiDang2001VN · 2018-06-22T17:50:42Z

I have reinstall via pip but just get ImportError again.
And I use Python 3.5.0 (for more specific: the IDLE show me [MSC v.1900 64 bit (AMD64)] on win32).

takanokage · 2018-06-22T17:56:34Z

@qianyizh in your case seems easy enough: install glew. Pretty much all of the dependencies that where installed on my build system have to be installed on the target system.

takanokage · 2018-06-22T17:58:33Z

@HaiDang2001VN yeah, I added yesterday binaries for the 64bit version of Python 2.7 and 3.5.
The one reason that I can think of that this might not work was a report by two other people before that sometimes pip is unable to place the binaries in site-packages. It's just that we've never gathered enough information to figure out why - I can't reproduce it locally.

qianyizh · 2018-06-22T18:02:06Z

Yeah, I know what to do. But it is not written in the document - so it is not obvious.
I think I have similar issue with my Window machine too, some DLL libraries are missing. I think it is because some dependent libraries are either not installed or incorrectly configured.

We should resolve this, either add a document page describing how to install these dependency libraries on various operating systems, or make open3d-python a standalone package that can directly pip install and use out of the box (like opencv-python).

Btw, why does open3d-python install opencv too? I don't think opencv is a dependent library of open3d?

takanokage · 2018-06-22T18:18:13Z

OpenCV - That's on me. I followed the script for installing the python dependencies and there opencv is mentioned as mandatory.

So you're saying that opencv is not actually required? In that case I can fix both the script and the pip package, just let me know.

syncle · 2018-06-29T07:54:25Z

OpenCV is not mandatory. It is optional for Reconstruction system.

takanokage · 2018-06-29T15:31:20Z

Are we including the reconstruction system in the Open3D python module?

syncle · 2018-06-29T18:48:09Z

Reconstruction system is high-level Python tutorial scripts, although the basic functions for the system are implemented as C++ API in Open3D. It is not true that reconstruction system can be included in Open3D Python module, because it is just series of Python tutorial script. The reconstruction system optionally uses Python API of OpenCV, but OpenCV is not necessary to run Open3D python module.

In summary, open3d-python don't have to install opencv. Instead, it could be in util/scripts/install-deps-python.sh as a one of optional Python package.

syncle · 2018-06-29T18:57:13Z

We should resolve this, either add a document page describing how to install these dependency libraries on various operating systems, or make open3d-python a standalone package that can directly pip install and use out of the box (like opencv-python).

I agree. Can we make a standalone package? I think this would be much convenient than writing a long document case by case. As @qianyizh mentioned, I also encountered the similar error:

>>> import open3d
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jaesikpa/anaconda/lib/python3.6/site-packages/open3d/__init__.py", line 11, in <module>
    from open3d.macos import *
  File "/Users/jaesikpa/anaconda/lib/python3.6/site-packages/open3d/macos/__init__.py", line 7, in <module>
    globals().update(importlib.import_module('open3d.macos.open3d').__dict__)
  File "/Users/jaesikpa/anaconda/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: dlopen(/Users/jaesikpa/anaconda/lib/python3.6/site-packages/open3d/macos/open3d.so, 2): Library not loaded: /usr/local/opt/jpeg/lib/libjpeg.9.dylib
  Referenced from: /Users/jaesikpa/anaconda/lib/python3.6/site-packages/open3d/macos/open3d.so
  Reason: Incompatible library version: open3d.so requires version 13.0.0 or later, but libjpeg.9.dylib provides version 12.0.0

I know this is dependency problem, but someone just want to use Open3D out-of-box.

qianyizh · 2018-07-03T17:36:02Z

This is burning. I just tried on my Ubuntu machine, and this is what I got:

> pip install open3d-python
Collecting open3d-python
  Downloading https://files.pythonhosted.org/packages/2d/b3/3c1a6afa37f9ad2c85ab2c2abd249432bab06fb8e00ab94f8c4cd060b234/open3d_python-0.1.5.5-py2.py3-none-any.whl (7.7MB)
    100% |████████████████████████████████| 7.8MB 6.8MB/s 
Requirement already satisfied: numpy in /home/qianyizh/anaconda3/lib/python3.6/site-packages (from open3d-python) (1.14.3)
Requirement already satisfied: matplotlib in /home/qianyizh/anaconda3/lib/python3.6/site-packages (from open3d-python) (2.2.2)
Collecting opencv-python (from open3d-python)
  Using cached https://files.pythonhosted.org/packages/7c/78/bc95e29057e80d1da71912426113e85f20752b1031193f51750322354937/opencv_python-3.4.1.15-cp36-cp36m-manylinux1_x86_64.whl
Requirement already satisfied: cycler>=0.10 in /home/qianyizh/anaconda3/lib/python3.6/site-packages (from matplotlib->open3d-python) (0.10.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /home/qianyizh/anaconda3/lib/python3.6/site-packages (from matplotlib->open3d-python) (2.2.0)
Requirement already satisfied: python-dateutil>=2.1 in /home/qianyizh/anaconda3/lib/python3.6/site-packages (from matplotlib->open3d-python) (2.6.1)
Requirement already satisfied: pytz in /home/qianyizh/anaconda3/lib/python3.6/site-packages (from matplotlib->open3d-python) (2017.2)
Requirement already satisfied: six>=1.10 in /home/qianyizh/anaconda3/lib/python3.6/site-packages (from matplotlib->open3d-python) (1.11.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /home/qianyizh/anaconda3/lib/python3.6/site-packages (from matplotlib->open3d-python) (1.0.1)
Requirement already satisfied: setuptools in /home/qianyizh/anaconda3/lib/python3.6/site-packages (from kiwisolver>=1.0.1->matplotlib->open3d-python) (36.5.0.post20170921)
Installing collected packages: opencv-python, open3d-python
Successfully installed open3d-python-0.1.5.5 opencv-python-3.4.1.15

Python 3.6.5 |Anaconda custom (64-bit)| (default, Apr 29 2018, 16:14:56) 
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import open3d
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/qianyizh/anaconda3/lib/python3.6/site-packages/open3d/__init__.py", line 9, in <module>
    from open3d.linux import *
  File "/home/qianyizh/anaconda3/lib/python3.6/site-packages/open3d/linux/__init__.py", line 7, in <module>
    globals().update(importlib.import_module('open3d.linux.open3d').__dict__)
  File "/home/qianyizh/anaconda3/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: /home/qianyizh/anaconda3/lib/python3.6/site-packages/open3d/linux/open3d.so: undefined symbol: _Py_ZeroStruct

On my OSX:

Python 3.6.4 |Anaconda custom (64-bit)| (default, Jan 16 2018, 12:04:33) 
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import open3d
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/qianyizh/anaconda/envs/py36/lib/python3.6/site-packages/open3d/__init__.py", line 11, in <module>
    from open3d.macos import *
  File "/Users/qianyizh/anaconda/envs/py36/lib/python3.6/site-packages/open3d/macos/__init__.py", line 7, in <module>
    globals().update(importlib.import_module('open3d.macos.open3d').__dict__)
  File "/Users/qianyizh/anaconda/envs/py36/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: dlopen(/Users/qianyizh/anaconda/envs/py36/lib/python3.6/site-packages/open3d/macos/open3d.so, 2): Library not loaded: /usr/local/opt/glew/lib/libGLEW.2.1.dylib
  Referenced from: /Users/qianyizh/anaconda/envs/py36/lib/python3.6/site-packages/open3d/macos/open3d.so
  Reason: image not found

On my windows: never succeeded.

We need to hot fix this. @takanokage can you look into this and give a daily update about it? If you are on vacation, assign someone else (Jaesik or me) so we know the ownership.

takanokage · 2018-07-03T17:55:18Z

I see there's a mismatch between python installations. It would seem that you're running a custom Python installation, this is why it's not able to import Open3D.

Open3D as it is today is only supported with the standard Python installation. That was the specification of the pip package ticket when I worked on it.

Open3D is not supported on Anaconda Python - this is not a bug, it is a feature request.
This week I'm 100% working on another project, adding support for Anaconda, or anything else for that matter, will have to wait.

qianyizh · 2018-07-03T18:06:53Z

Thanks for the update. Agree this is a feature that has not been built yet. And we should build it.

I am more worried about the ownership of this issue since I know you will be on leave since next week. If you can't work on this this week either, then it probably makes more sense for me or @syncle to just take the ownership until you come back from leave.

takanokage · 2018-07-03T18:07:38Z

Open3D is supported on Ubuntu/macOS/Windows only on a standard/native Python distribution, not AnaConda.
pip install open3d-python was tested and found to be working out of the box with:

Windows, python installed from https://www.python.org/downloads/, 2.7 & 3.5 32bit and 64bit
MacOS, system python 2.7
Ubuntu, system python 2.7 and python 3.5 installed through apt.

For any other use cases please open appropriate feature requests and close this.

Explanation:
It is possible to invoke pip with package managers such as conda however this is a non-standard way of installing packages on conda. Open3D is supported on only native Python installations and will not work if installed with pip from conda just because they use different paths for the pip packages.

takanokage · 2018-07-03T18:13:33Z

@qianyizh
We have a workaround for this: native Python installation.
I would recommend reducing the urgency from red hot to highly desirable ;). I am hoping that this workaround can help alleviate the problem.

I think a lot of this confusion was caused by the fact that so far we've assumed that all Python installations are created equal. That is not true. We need a list of all the package managers that manage Python in their own unique way and after that decide which of these we want to support.

qianyizh · 2018-07-03T18:26:03Z

Good. I think the easiest fix for now is to add this as a "note" in the document of:
http://www.open3d.org/docs/tutorial/Basic/python_interface.html#install-open3d-from-pypi

Open3D is supported on Ubuntu/macOS/Windows only on a standard/native Python distribution, not AnaConda.
pip install open3d-python was tested and found to be working out of the box with:

* Windows, python installed from https://www.python.org/downloads/, 2.7 & 3.5 32bit and 64bit
* MacOS, system python 2.7
* Ubuntu, system python 2.7 and python 3.5 installed through apt.

I will test native builds tonight and if it works I am up for closing this ticket for now (and open a new request for conda-backed python!).

qianyizh · 2018-07-03T18:28:29Z

As a hindsight, I think the confusion came from the lack of documentation. I learnt it just now (that the pip package does not work with conda-backed python) and I am pretty sure I am an experienced user of Open3D :). So we should have it documented.

syncle · 2018-07-03T18:46:30Z

Great. I will submit PR to update document accordingly.

takanokage · 2018-07-03T18:55:11Z

@qianyizh I don't think anybody realized that the issue was different Python installations until you posted the error message showing an anaconda path. This wasn't just our documentation lacking. In fact this is a subtle issue of which I am going to assume most Python users (including ourselves) may not be aware of: there several ways to install Python and these are not necessarily compatible between each other in every aspect.

syncle · 2018-07-06T19:37:28Z

To add some more detail, I have tested with a fresh Ubuntu 16.04 and encountered the similar external library issue aforementioned. I used naive pip, no anaconda installed.

pip distribution runs out-of-box

if all the required dependency are already installed (we have already opened issue for this)
if the installed external library version matches with the one used for buildind pip open3d distribution. (For example, even if libpng is already installed in the system, if it is not libpng16, it cannot import. This was my case when I tried with fresh ubuntu.)

However, this is not the case for a user (consider a user who didn't visit our github repo and didn't know about install-deps-ubuntu/mac.sh) who just want to install open3d via pip. That's why we have many corner cases.

One quick remedy

add description in pypi page that you would need to visit github and download/run shell script before try pip install

Better remedy

Fix pip package that can explicitly control dependencies.

qianyizh · 2018-07-08T00:41:20Z

This is very bad. We need hot fix. I will take over the ownership.

Travis346 · 2018-09-14T19:11:04Z

Any update on this? I get the same issue:
Windows 10 (x64), Python 3.7 (x64), native pip install gives

yxlao · 2018-09-14T20:12:24Z

@Travis346 Python 3.7 is not supported in our pip package. If you unzip the pip package, you'll see binaries for Python 2.7, 3.5 and 3.6.

├── __init__.py
├── linux
│   ├── __init__.py
│   ├── open3d.cpython-35m-x86_64-linux-gnu.so
│   ├── open3d.cpython-36m-x86_64-linux-gnu.so
│   ├── open3d.so
│   └── readme.txt
├── macos
│   ├── __init__.py
│   ├── open3d.cpython-35m-darwin.so
│   ├── open3d.cpython-36m-darwin.so
│   ├── open3d.so
│   └── readme.txt
└── win32
    ├── 32b
    │   ├── __init__.py
    │   ├── open3d.cp35-win32.pyd
    │   ├── open3d.cp36-win32.pyd
    │   └── open3d.pyd
    ├── 64b
    │   ├── __init__.py
    │   ├── open3d.cp35-win_amd64.pyd
    │   ├── open3d.cp36-win_amd64.pyd
    │   └── open3d.pyd
    ├── __init__.py
    └── readme.txt

The quick fix would be to compile from source, or, use the 2.7, 3.5 and 3.6 Python version. Later we may add Python 3.7 support as well.

BTW, we now have a repository with CI jobs that test pip and conda installations: https://github.com/IntelVCL/Open3D_Python_CI/blob/master/appveyor.yml. They're passing for 2.7, 3.5, 3.6 on Windows.

syncle · 2018-09-28T03:35:04Z

@Travis346: the dynamic library link issue is addressed with Open3D v0.3. Can you try out this? @HaiDang2001VN: as a note, from open3d v0.2, we renamed py3d to open3d.

gtorpiano · 2018-11-14T00:09:35Z

Hi, is there a fix for DLL issue? Does it work on python3.7? Thanks

yxlao · 2018-11-14T01:22:52Z

Currently, we don't have pre-compiled library for python3.7 yet. To use Open3D with python3.7, please compile from source.

We do have plan to add pre-compiled library for python3.7 in the future -- potentially after the automatic packaging CI is setup.

syncle · 2018-12-17T19:17:26Z

@HaiDang2001VN @gtorpiano @Travis346: we have resolved many issues for pip package. Could you try again and share the status?
@yxlao: how about we add a support for Python 3.7?

yxlao · 2018-12-18T08:05:39Z

Potentially we can add it in v0.5 release, but it would be a manual add. For v0.6 when we have the automatic packaging, we should definitely support it. What do you think @syncle ?

syncle · 2018-12-18T08:29:46Z

@yxlao: I agree. Let's do that for the next release.

yxlao · 2019-02-19T19:25:22Z

Python 3.7 supported has been added since v0.5.0

syncle added bug labels Jun 26, 2018

takanokage self-assigned this Jul 1, 2018

syncle mentioned this issue Jul 2, 2018

installing open3d python #425

Closed

qianyizh mentioned this issue Jul 3, 2018

Compilation does not work out-of-the-box when external libraries are not installed #407

Closed

syncle mentioned this issue Jul 3, 2018

Documentation #426

Merged

syncle mentioned this issue Jul 4, 2018

Anaconda support #430

Closed

qianyizh self-assigned this Jul 6, 2018

qianyizh added the high priority label Jul 8, 2018

qianyizh unassigned takanokage Jul 8, 2018

qianyizh removed the development label Jul 8, 2018

graphedge mentioned this issue Jan 8, 2019

Please test Open3d with Windows 10 Anaconda Jupyter and/or CLI #764

Closed

yxlao closed this as completed Feb 19, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Failed when import py3d on Windows 10 #408

Failed when import py3d on Windows 10 #408

HaiDang2001VN commented Jun 22, 2018

takanokage commented Jun 22, 2018

HaiDang2001VN commented Jun 22, 2018

takanokage commented Jun 22, 2018

qianyizh commented Jun 22, 2018

HaiDang2001VN commented Jun 22, 2018

takanokage commented Jun 22, 2018

takanokage commented Jun 22, 2018

qianyizh commented Jun 22, 2018

takanokage commented Jun 22, 2018 •

edited

syncle commented Jun 29, 2018

takanokage commented Jun 29, 2018

syncle commented Jun 29, 2018

syncle commented Jun 29, 2018

qianyizh commented Jul 3, 2018

takanokage commented Jul 3, 2018

qianyizh commented Jul 3, 2018

takanokage commented Jul 3, 2018

takanokage commented Jul 3, 2018

qianyizh commented Jul 3, 2018

qianyizh commented Jul 3, 2018

syncle commented Jul 3, 2018

takanokage commented Jul 3, 2018

syncle commented Jul 6, 2018

qianyizh commented Jul 8, 2018

Travis346 commented Sep 14, 2018

yxlao commented Sep 14, 2018 •

edited

syncle commented Sep 28, 2018

gtorpiano commented Nov 14, 2018

yxlao commented Nov 14, 2018

syncle commented Dec 17, 2018

yxlao commented Dec 18, 2018

syncle commented Dec 18, 2018

yxlao commented Feb 19, 2019

Failed when import py3d on Windows 10 #408

Failed when import py3d on Windows 10 #408

Comments

HaiDang2001VN commented Jun 22, 2018

takanokage commented Jun 22, 2018

HaiDang2001VN commented Jun 22, 2018

takanokage commented Jun 22, 2018

qianyizh commented Jun 22, 2018

HaiDang2001VN commented Jun 22, 2018

takanokage commented Jun 22, 2018

takanokage commented Jun 22, 2018

qianyizh commented Jun 22, 2018

takanokage commented Jun 22, 2018 • edited

syncle commented Jun 29, 2018

takanokage commented Jun 29, 2018

syncle commented Jun 29, 2018

syncle commented Jun 29, 2018

qianyizh commented Jul 3, 2018

takanokage commented Jul 3, 2018

qianyizh commented Jul 3, 2018

takanokage commented Jul 3, 2018

takanokage commented Jul 3, 2018

qianyizh commented Jul 3, 2018

qianyizh commented Jul 3, 2018

syncle commented Jul 3, 2018

takanokage commented Jul 3, 2018

syncle commented Jul 6, 2018

qianyizh commented Jul 8, 2018

Travis346 commented Sep 14, 2018

yxlao commented Sep 14, 2018 • edited

syncle commented Sep 28, 2018

gtorpiano commented Nov 14, 2018

yxlao commented Nov 14, 2018

syncle commented Dec 17, 2018

yxlao commented Dec 18, 2018

syncle commented Dec 18, 2018

yxlao commented Feb 19, 2019

takanokage commented Jun 22, 2018 •

edited

yxlao commented Sep 14, 2018 •

edited