Deadlock loading usd

[rsodre]

Description of Issue

The new Maxon Cinema 4D R23 introduces USD support, as a shared library.
A custom plugin for Cinema 4D (macOS) that also includes usd freezes the app when loading.
Pausing the debugger while the app is frozen, it appears that USD is in a deadlock.
My C4D plugin is a dynamic library (.xlib), USD was built as static library and linked with it.
I changed my USD internal namespace to pxrEphere, that way we can see on the call stack (attached) at line 28 what seems to be C4D's pxrInternal::AddImage function being calledby ImageLoader(), and from there on my own pxrEphere functions are called, until it gets into ImageLoader() again, and locks.
The call stack is from a Debug build, but the same happens on Release.

I wonder what can be done avoid that situation.
Could it be related to tbb?
Seems that my plugin is loading the usd dynamic library, why would it fdo that if it already includes usd?

I tried building USD as a shared library and deploying with my plugin, the problem persists.
Removing C4D's usd module, the plugin loads successfully.
On previous versions of C4D that do not have usd, the plugin loads successfully.

Steps to Reproduce

1. Build a Cinema 4D plugin, linked with USD as a monolithic static library
2. Run C4D from XCode, the app freezes

System Information (OS, Hardware)

macOS 10.15.6
iMac 4K 2017
(no problem on Windows 10)

Package Versions

USD 0.20.2
Cinema 4D SDK R23.008

Build Flags

PXR_BUILD_MONOLITHIC=ON
PXR_INTERNAL_NAMESPACE="pxrEphere_v${PXR_MAJOR_VERSION}_${PXR_MINOR_VERSION}"

[jilliene]

Filed as internal issue #USD-6375

[rsodre]

Filed as internal issue #USD-6375

I'm not sure what to expect now, should I just wait?
Is there a way to track this internal issue?

My plugin have all USD features removed, and that's one of the big new features of the upcoming release.

[sunyab]

Hi @rsodre, that issue number is for our own internal processes, this issue would be the best way for you to keep track of any updates and progress.

My initial suspicion is that you may be building your plugin against a different version of TBB than the version used by Cinema 4D's own USD module. We've seen similar issues with other DCCs in the past. If the C4D SDK provides a version of TBB or if you can determine what version they're using, it might help to build your version of USD against that instead.

We do not have Cinema 4D available internally so our ability to offer support here is limited. You may want to contact Maxon to see if they can provide more assistance.

[rsodre]

Thanks for the feedback @sunyab, I will check both tbb versions.

[rgigante]

Cinema 4D R23 RC uses TBB 2019 u3 but next releases will switch to 2020 u2.

Cheers, R

[ikolev21]

We resolved this by switching our plugin to refer to the copy of libtbb.dylib included with C4D's USD module, using install_name_tool -change

As our plugin runs on 3 platforms and on 3 hosts (3dsMax/Maya/C4D), our approach to TBB usage is to rely on its full backward compatibility: we compile/link against the oldest version that does the job for us (4.4) and expect that the hosts provide newer versions.
In case of C4D we didn't do that because C4D doesn't include TBB in its main application, though some of its external modules do include their own copy of TBB. Similar to the other modules, we included our (old 4.4) copy of TBB.

But this was no problem until now, the copies of TBB carried by C4D's modules haven't interfered in any harmful way. And the new USD module in R23 didn't change that.

The source of the problem here is probably the fact that our plugin links to our own static copy of USD, which causes USD to be loaded twice, along with two different copies of TBB (and maybe the version mismatch is not important).
So, linking against C4D's copy of TBB resolved the TBB deadlock problem, but we have now a new duplicate symbol problem, which I opened a new issue about: #1479