The source directory /tmp/pip-install-xx/bpy-cuda does not appear to contain CMakeLists.txt

[jl-DaDar]

Describe the bug
I am trying to install this on linux ubuntu on ec2.
I am gettin g the following error

cmake -DWITH_PLAYER=OFF -DWITH_PYTHON_INSTALL=OFF -DWITH_PYTHON_MODULE=ON -DPYTHON_VERSION=3.7 -DWITH_CYCLES_CUDA_BINARIES=ON -DWITH_AUDASPACE=OFF -S/tmp/pip-install-5xcwp94i/bpy-cuda/build/temp.linux-x86_64-3.7/blender -B/tmp/pip-install-5xcwp94i/bpy-cuda/build/temp.linux-x86_64-3.7/build CMake Error: The source directory "/tmp/pip-install-5xcwp94i/bpy-cuda" does not appear to contain CMakeLists.txt. Specify --help for usage, or press the help button on the CMake GUI. error: command 'cmake' failed with exit status 1

To Reproduce
Steps to reproduce the behavior:

1. Install prerequisites
2. run pip3 install bpy-cuda

Expected behavior
I was hoping it would work

Desktop (please complete the following information):

• OS: Amazonlinux2 on gdx4n instance with GPU
• Docker Container: nvidia/cudagl:10.2-runtime-ubuntu18.04
  I am trying to build this within Docker container

btw, it takes super long to install. It seems to be very slow at downloading stuff from blender git and svn

[TylerGubala]

I have seen this happen before on systems that do not have a recent version of cmake.

In version 3.13.5, cmake added a new syntax, with -S <path-to-source> -B <path-to-build>

If you have a version less than that, the command generally gives you the error you are showing.

The time it takes to install is unfortunate but out of my control until I can publish wheel files to pypi, which is pending and increase to my file size upload limit.

What does cmake --version tell you? If it is less than 3.13.5, you will need to install a more recent version.

[jl-DaDar]

yes the cmake version is 3.10. I updated i to 3.17. The thing is that cmake 3.10 is installed during pip install bpy, so it would cause the installation to fail. If I try to install newer cmake before the command, cmake would still get overwritten by the older one.

anyways, I try to run it after i replace cmake, I get this error instead

cmake -DWITH_PLAYER=OFF -DWITH_PYTHON_INSTALL=OFF -DWITH_PYTHON_MODULE=ON -DPYTHON_VERSION=3.7 -DWITH_CYCLES_CUDA_BINARIES=ON -DWITH_AUDASPACE=OFF -S/tmp/pip-install-_mavrfpw/bpy-cuda/build/temp.linux-x86_64-3.7/blender -B/tmp/pip-install-_mavrfpw/bpy-cuda/build/temp.linux-x86_64-3.7/build CMake Error: Could not find CMAKE_ROOT !!! CMake has most likely not been installed correctly. Modules directory not found in /usr/share/cmake-3.17 CMake Error: Error executing cmake::LoadCache(). Aborting.

[TylerGubala]

Ah, okay. I changed the bpy-build requirement to install cmake>=3.13.5

Also, it seems like your installation of cmake is corrupt by that error message. You might want to uninstall and reinstall cmake.

If it's more convenient to you you could install from wheel here:

https://drive.google.com/drive/u/0/folders/18HFAbqoPBF6ItYrwbeQB-xSEbq3VudsI

python3.7 -m pip install -U pip
python3.7 -m pip install bpy_cuda-2.82-cp37-cp37m-linux_x86_64.whl && bpy_post_install

[jl-DaDar]

I did a fresh install and i can find bpy_post_install, but when I try to import bpy, libraries are missing. I then try to attemtp to add one by one

apt-get install libtbb2 libjemalloc1 libfftw3-3

then it ask for

import bpy
Traceback (most recent call last):
File "", line 1, in
ImportError: libosdGPU.so.3.4.0: cannot open shared object file: No such file or directory

this one seems to be from blender, I am not sure how I can get it.

[jl-DaDar]

I am quite stuck now.
I tried on a container that had previously failed to install bpy, so that I could have all the libraries I need. Then I try to import bpy

import bpy
Color management: using fallback mode for management
Color management: Error could not find role data role.
BLT_lang_init: 'locale' data path for translations not found, continuing
Color management: scene view "Filmic" not found, setting default "Standard".
bpy: couldn't find 'scripts/modules', blender probably wont start.
Freestyle: couldn't find 'scripts/freestyle/modules', Freestyle won't work properly.
ModuleNotFoundError: No module named 'bpy_types'
ModuleNotFoundError: No module named 'bpy_types'
ERROR (bpy.rna): /home/tylergubala/Code/Python/blenderpy/build/temp.linux-x86_64-3.7/blender/source/blender/python/intern/bpy_rna.c:7083 pyrna_srna_ExternalType: failed to find 'bpy_types' module
ModuleNotFoundError: No module named 'bpy_types'
ModuleNotFoundError: No module named 'bpy_types'
ERROR (bpy.rna): /home/tylergubala/Code/Python/blenderpy/build/temp.linux-x86_64-3.7/blender/source/blender/python/intern/bpy_rna.c:7083 pyrna_srna_ExternalType: failed to find 'bpy_types' module
ModuleNotFoundError: No module named 'bpy_types'
ERROR (bpy.rna): /home/tylergubala/Code/Python/blenderpy/build/temp.linux-x86_64-3.7/blender/source/blender/python/intern/bpy_rna.c:7083 pyrna_srna_ExternalType: failed to find 'bpy_types' module
ModuleNotFoundError: No module named 'bpy_types'
ERROR (bpy.rna): /home/tylergubala/Code/Python/blenderpy/build/temp.linux-x86_64-3.7/blender/source/blender/python/intern/bpy_rna.c:7083 pyrna_srna_ExternalType: failed to find 'bpy_types' module
ModuleNotFoundError: No module named 'bpy_types'
ERROR (bpy.rna): /home/tylergubala/Code/Python/blenderpy/build/temp.linux-x86_64-3.7/blender/source/blender/python/intern/bpy_rna.c:7083 pyrna_srna_ExternalType: failed to find 'bpy_types' module
F0525 18:07:18.204803 18296 utilities.cc:322] Check failed: !IsGoogleLoggingInitialized() You called InitGoogleLogging() twice!
*** Check failure stack trace: ***
@ 0x7f258cd3521d google::LogMessage::Fail()
@ 0x7f258cd370c2 google::LogMessage::SendToLog()
@ 0x7f258cd34d5e google::LogMessage::Flush()
@ 0x7f258cd37a69 google::LogMessageFatal::~LogMessageFatal()
@ 0x7f258cd3d922 google::glog_internal_namespace_::InitGoogleLoggingUtilities()
@ 0x7f258c69fc9d libmv_initLogging
@ 0x7f258bcf2a92 main_python_enter
@ 0x7f258c19c10d bpy_module_delay_init
@ 0x7f258c19c16d dealloc_obj_dealloc
@ 0x5b9a27 PyDict_SetItem
@ 0x5b9eac PyDict_SetItemString
@ 0x533d39 PyModule_AddObject
@ 0x636b64 _PyImport_LoadDynamicModuleWithSpec
@ 0x636d7f (unknown)
@ 0x5d83f7 _PyMethodDef_RawFastCallDict
@ 0x5d85ba PyCFunction_Call
@ 0x552810 _PyEval_EvalFrameDefault
@ 0x54b302 _PyEval_EvalCodeWithName
@ 0x5d8bd2 _PyFunction_FastCallKeywords
@ 0x54a880 (unknown)
@ 0x551c08 _PyEval_EvalFrameDefault
@ 0x5d88dc _PyFunction_FastCallKeywords
@ 0x54de9e _PyEval_EvalFrameDefault
@ 0x5d88dc _PyFunction_FastCallKeywords
@ 0x54dd08 _PyEval_EvalFrameDefault
@ 0x5d88dc _PyFunction_FastCallKeywords
@ 0x54dd08 _PyEval_EvalFrameDefault
@ 0x5d88dc _PyFunction_FastCallKeywords
@ 0x54dd08 _PyEval_EvalFrameDefault
@ 0x5d9bf6 _PyFunction_FastCallDict
@ 0x5db3e4 (unknown)
@ 0x5db84d _PyObject_CallMethodIdObjArgs
Aborted (core dumped)

[TylerGubala]

In truth I am not sure what is going on here. Maybe the Amazon Ubuntu image is different somehow from what I am using. When I import bpy after installing the wheel in a freshly made venv on Ubuntu Server 18.04 LTS I only get

/run/user/1000/gvfs/ non-existent directory

Then the import succeeds. What you are showing above is an error commonly associated with not having the 2.82 directory in the right place. On Ubuntu, it needs to be in the site-packages folder.

What does

find . -type d -name "2.81" -print

Tell you? Note that your current working directory should be the python root of the environment you are installing bpy to. I highly recommend using venv for this purpose until we figure out what is going wrong in your environment.

Also, you might have to ask around on the Blender Dev talk and make a build manually on your system of choice, following the wiki

[jl-DaDar]

I didn't find anything with 2.81, i assume you want 2.82

root@67626b8743a3:/# find . -type d -name "2.82" -print
find: './proc/3579/task/3579/net': Invalid argument
find: './proc/3579/net': Invalid argument
find: './proc/4413/task/4413/net': Invalid argument
find: './proc/4413/net': Invalid argument
./usr/local/bin/2.82

I am doing this in docker, so I didn't use venv, but I will try.

to summarize:

1. right now if I try to build using pip install bpy-cuda I get
   CMake Error at CMakeLists.txt:834 (message): Cycles OSL requires WITH_LLVM, the library may not have been found. Configure LLVM or disable WITH_CYCLES_OSL
   and it doesn't get installed properly.

2. if I use the wheel package you gave me, it could not find many of the .so files. some I could fix by apt-get install, but i couldn't find libosdGPU.so.3.4.0

3. So now I basically try step1 first (even though it fails) to get all the .so file I need and then install the wheel package you gave me. but when I i try to import bpy in python3.7 console, I get

bpy: couldn't find 'scripts/modules', blender probably wont start. Freestyle: couldn't find 'scripts/freestyle/modules', Freestyle won't work properly. ModuleNotFoundError: No module named 'bpy_types' ModuleNotFoundError: No module named 'bpy_types' ERROR (bpy.rna): /home/tylergubala/Code/Python/blenderpy/build/temp.linux-x86_64-3.7/blender/source/blender/python/intern/bpy_rna.c:7083 pyrna_srna_ExternalType: failed to find 'bpy_types' module ModuleNotFoundError: No module named 'bpy_types' ModuleNotFoundError: No module named 'bpy_types' ERROR (bpy.rna): /home/tylergubala/Code/Python/blenderpy/build/temp.linux-x86_64-3.7/blender/source/blender/python/intern/bpy_rna.c:7083 pyrna_srna_ExternalType: failed to find 'bpy_types' module

anyways, thanks so much for you help. I really appreciate it. really hope I can get this to work.

[jl-DaDar]

looks like I managed to get this to work by copying 2.82 to site-packages, like you suggested.
just 2 more issues:

1. When I run a blender script, it will print out "Error: Not freed memory blocks: 8, total unfreed memory 0.010223 MB" does it mean it has memory leak?

2. It also prints out '/usr/local/lib/python3.7/dist-packages/2.82/python/lib/python3.7/site-packages/libextern_draco.so' does not exist, draco mesh compression not available

are there way to resolve these 2 issues?

[TylerGubala]

There is a bit of something concerning that I noticed, when you ran the find bash command, well first, it says you are root, and I just hope you aren't running everything as root all the time...

Secondly, you appear to be running the find command from the home directory, and the 2.82 directory that find found was in /usr/local/bin it appears.

I didn't find anything with 2.81, i assume you want 2.82
root@67626b8743a3:/# find . -type d -name "2.82" -print
[...]
./usr/local/bin/2.82

That would normally be the place that Blender 3D the application would put the 2.82 directory, and I'm just nervous that you would be copying that instead of trying to run bpy_post_install, which should be available after the install from the .whl file, and will copy the correct 2.82 directory (the one that ships with the python module) into the proper place.

The full installation process in a fresh environment should look like this after downloading the .whl file (regardless if in Docker or venv or whatever environment management solution):

pip install bpy-2.82-cp37-cp37m-linux_x86_64.whl && bpy_post_install

That should really be about it.

1. When I run a blender script, it will print out "Error: Not freed memory blocks: 8, total unfreed memory 0.010223 MB" does it mean it has memory leak?

This is a known issue. I think it does mean it has a memory leak! You should check on Blender Dev talk, a place where Blender Developers get together and talk about how people use Blender and bugs, etc.

2. It also prints out '/usr/local/lib/python3.7/dist-packages/2.82/python/lib/python3.7/site-packages/libextern_draco.so' does not exist, draco mesh compression not available

I have no idea about this one! This does not appear in my Linux machine. Maybe it's something unique to your system. Does it cause any errors when manipulating mesh data directly?

[jl-DaDar]

I am doing everything in docker container, so I have root priviledge by default. But you are right, I shouldn't run everything as root.

Right now it doesn't work as clean as you suggested. If I install your wheel package and do bpy_post_install, 1)directory is still not in the proper place and. 2) it is still missing a lot of .so file. I have to hunt down packages one by one using apt-install or attempt to build from source(even though building from source fails, it does create the .so files I need) and that's really how I got it to work right now. Built once from source, it fails, then install your wheel package and run bpy_post_install, and then move directory.

It also prints out '/usr/local/lib/python3.7/dist-packages/2.82/python/lib/python3.7/site-packages/libextern_draco.so' does not exist, draco mesh compression not available

I only get this when export scene as gltf, which uses draco to compress mesh if option set to True. When I use blender as desktop app it doesn't have this issue.

[TylerGubala]

I think it would be interesting to see if the install_deps.sh command resolves your need for the .so files.

I don't think that bpy.so is portable across machines. I think I'll bring that up in Blender Dev talk and see if I get a response.

That may help with the Draco issue, though I'm not sure if I have that. How are you exporting as .gltf? What is that command? I would like to test on my Linux box and see if I have the same thing.

[TylerGubala]

@jl-DaDar If you are still interested in using this repo, I have been debugging usage inside of Docker finally and I don't know if I am just starting to go cross-eyed but it's almost starting to look feasible.

I would really appreciate any insight from someone who is actually using bpy.so that way.

[TylerGubala]

Try installing the wheel file that is supplied in the Releases page.

If that does not work for you, then please post what doesn't work in a new issue. Thanks.