Python bindings not importing on OSX #19
Comments
|
I wrote a little script to symlink the dylibs to the name so. That works. You need to symlink, because of the embedded rpath stuff. There's probably a way to get cmake to generate .so instead of .dylib ... ? |
|
Thanks Nick; that gets me a bit further-- when importing _tf, the Py_IsInitialized() axiom fails, and then the interpreter hangs. The boost::python examples at https://github.com/TNG/boost-python-examples seem to build and run ok. I am on Python 2.7.12; will try 2.7.5 and see if it makes a difference. |
|
You might try running python -v, and reading the dynamic loading spew (thanks to sunyab for this tip). Crashes will occur if same-named dylibs hiding in other search paths are taking precedence over the ones you linked against. Maybe there's already boost_python elsewhere in your system... |
|
Cool, that tip got me in the right direction-- running with DYLD_PRINT_LIBRARIES=1 showed me that Tf was linked against the wrong Python library. After specifying the desired python locations to cmake, I'm able to import _tf. I'll leave this open to track build support for .so's on OSX, so that we don't have to make the symlinks. Thanks again! |
|
Filed as internal issue #135883. |
|
So the solution is that the Python lib generation script should specify the ".so" suffix. diff --git a/cmake/macros/Public.cmake b/cmake/macros/Public.cmake
|
Hi, I also got this problem, I'm using static boost.python with the Python 2.7.12 from brew, but the TBB came from Maya 2017. The workaround switch for TBB "-Wl,-Bsymbolic-functions" didn't work on my Clang 7.3. The simple python script always hangs at the libtbbmalloc. Is there any ideas about this case ? Thanks very much. |
|
When you configured cmake the first time, I assume you pointed to the tbb in Maya, and that's how it was discovered? Also I assume you are attempting to be able to use Maya's Python bindings, and USD python bindings simultaneously? If yes to both, I think you should open a new issue: Using libtbbmalloc on macOS causes hang. I'm able to use the USD Python bindings because I don't have libtbbmalloc in my build at all. Also, I don't have to interoperate with Maya's bindings, so I got a get out of jail free card. |
|
As a datapoint, I have libtbbmalloc on OSX, but no Maya, and am able to use On Fri, Sep 30, 2016 at 6:43 PM, Nick Porcino notifications@github.com
|
|
@meshula Yes, that's what my setup, use the single tbb from Maya and also the external bindings. But my problem seems a bit more complicated, I'm using the 10.11 but still use the OSX 10.9 SDK for the all dependencies, but the Python comes from Brew was compiled with 10.11 the latest SDK, I'm not sure if that's the issue, the problem in Python. I will try again. Thank you very much guys. |
|
As you suspect, you'll need to conform your build to Maya completely and not mix-and-match SDKs... By conform it, I mean, compile vs. the same SDKs as Maya. FWIW, I'm personally building against system Python, the one in /System/Library/Frameworks, and that works, Maya seems to like it okay. |
|
I fixed this at my local in a weird way. TF_REGISTRY_FUNCTION(TfDebug) {
TfType::Define< TfDebugSymbolsChangedNotice,
TfType::Bases<TfNotice> >();
TfType::Define< TfDebugSymbolEnableChangedNotice,
TfType::Bases<TfNotice> >();I debugged the whole routine for initialization of Tf, it always reported the TfDebugSymbolsChangedNotice was not registered, so I move the code from debugNotice.cpp to debugCodes.cpp before the lines of TF_DEBUG_ENVIRONMENT_SYMBOL, then it worked. Not sure this is the correct fix but there is the proper order to be handled during the initialization. At least my local Python binding is working now. Thanks very much. |
|
Wow, congratulations on working out a very tricky dependency ordering problem, I haven't seen that problem before. I think that is worth tracking as an issue of its own. |
|
Hey @zhoub - I'm now seeing this same issue, and I really appreciate that you got it working, but I'm having trouble understanding what your fix was... So you removed the above code from |
|
Hi @drwave This is m commit https://github.com/zhoub/USD/commit/67200e58a7a1df276bc4caca00c04fabcc63ceb5 debugNotice.cpp keep it unchanged, then copy the lines to the debugCodecs.cpp Then just recompile it. When you're trying to load module it might report the warning about duplicated importing but it should work. To be honest this is absolutely not a good fix, ideally this problem should not even exist, also the logic of PyUSD is so complicated, maybe in future they will get this fixed. If you still have problems please let me know. |
|
Thanks - that’s exactly what I needed. Sorry, I’m still at the beginning stages of learning how to navigate github… I’m hoping someone on the core USD team can give me some insight into what’s going on.
|
|
Huge thanks to all for this. Combining:
I'm up and running (at least to run the HelloWorld tutorial) on my MacBook! |
|
Sadly I can't get through the endToEnd example though. Crashes when dealing with variants. Logged Issue #78 |
|
@robpieke FWIW, the patches for DebugSymbols and DebugNotice are patched into dev. |
|
@meshula sorry, didn't quite get that. are you saying the patches have now been merged into dev? i'm not seeing this after a "git pull" |
|
My mistake. Only #67 got merged. |
|
Hi, After some trial and error I was able to compile USD on MacOS 10.13.4 (XCode 10 Beta) by following the instructions mentioned in the README with minor adjustments (update OpenImageIO/oiio to 1.8.12 and installing PySide2 as mentioned in #60 ). I did not use XCode directly but instead ran the recommended Unfortunately running I suspect that something similar might be going on as mentioned by OP. I am quite new to build systems so I don't fully understand the various solutions mentioned. Would anyone be able to help me out? As mentioned in the build log of USD I've added the following to my environment: My goal is to convert the AR QuickLook Gallery examples to USDA so I can inspect their file-structure. Many thanks in advance, Tim |
|
Annoyingly, I'm in the same boat and can't remember how I solved this before. It looks like lib/python/pxr/Usd/_usd.so ends up linking to /System/Library/Frameworks/Python.framework/Versions/2.7/Python instead of /usr/lib/... |
|
I guess you did and see That's normal, if you intend everything to purely system python. If you are running into PyInitialize problems, it could be due to having both a system python 2.7, and also a homebrew, or Maya version of python in the path and/or cross wiring site-packages for different python installations via the environment variable. Pretty sure that if you want to build against a non-system python, you need to invoke the python executable that corresponds to way you want to run things. So if you want to build in a Maya compatible way, I think you need to invoke the script from Maya's bundled python executable, not the system python... |
|
Hey Nick, yes I used otool exactly as you indicated. I'm trying to build using the homebrew python. At least part of the complication seems to be that /usr/local/bin/python doesn't exist (only /usr/local/bin/python2.7), and some part(s) of the USD build process explicitly (re)invoke "python" regardless what you actually used as the interpreter when launching the build script. I've fired off one more test with python->python2.7 and python-config->python-config2.7 symlinks manually added just before leaving the office, so will need to check on Monday if that did the trick. Fingers-crossed. |
|
I think that will work. It sounds like the build_usd.py script needs to detect the hosting python and dispatch to the appropriate python using a full path. |
|
Nope, another failure. Google suggests Homebrew+FindPythonLibs=worldOfPain. Supposedly it's better in CMake 3.12 with a more generic FindPython. |
|
(Runs off to read release notes.) The new find scripts do look promising. The namespaced imported targets are an excellent step forward. https://cmake.org/cmake/help/git-stage/module/FindPython2.html#module:FindPython2 |
|
Reopening to track @meshula excellent suggestion of at least documenting this better! |
|
Using the solution mentioned by @robpieke I was able to correctly build and run USD! Thanks 👍 I've documented my installation here: https://github.com/timvanscherpenzeel/gltf-to-usdz#development as part of ongoing research. I thought I would post it here as it perhaps would be useful if further documentation changes are done in the future. |
update the openimagiio version all changes are from PixarAnimationStudios#19 (comment)
The Python detection code appears to give incorrect answers on Ubuntu: the code thinks the Python library lives in /usr/lib, but it actually lives in a different directory according to the multiarch specification. The same issue would presumably occur on any distribution implementing that spec, like Debian. Since the problem this code was intended to solve primarily occurs on macOS, this fix simply skips over the Python detection on Linux. See #19 See #699 (Internal change: 1933560)
|
Hey, late to the party hahaha, but this is still not working by default in 2023 :( Any updated install instructions for MacOS? The build script looks quite different these days... |
|
Hi @buhrmann , the Python bindings ought to be working on MacOS without additional tweaks to build_usd.py. If you're still running into problems I'd suggest filing a new issue with info on your build and what problems you're running into. Thanks! |
I know that the OSX build is currently experimental, but just want to verify that I'm seeing the currently expected behavior wrto Python bindings:
Likely due to http://stackoverflow.com/questions/2488016/how-to-make-python-load-dylib-on-osx , but I haven't found the right combination of xcode build settings to build _tf as a bundle with an .so extension yet, if that's the expected fix.
Cheers!
-a.
The text was updated successfully, but these errors were encountered: