Still having build issues on OSX

[mctrafik]

Good day.

Thank you for the recent updates. I'm still unable to use the plugin on OSX. So far:

• With the compiled plugin: onnxruntime cannot be found. If create a symlink to 1.5 from 1.7 the app crashes.
• I tried compiling on my machine again. With the updated code and instructions I'm able to run

cmake .. \
-DobsLibPath=/Applications/OBS.app/Contents/Frameworks \
-DobsIncludePath="../../obs-studio/libobs" \
-DLIBOBS_INCLUDE_DIR="../../obs-studio/libobs" \
-DLIBOBS_LIB="../../obs-studio/build/libobs/libobs.dylib"

but I DO have to specify DLIBOBS... attributes which are absent in the README. Not sure what I'm missing in my setup.

• Running cmake --build . fails with :

cmake --build .
Scanning dependencies of target obs-backgroundremoval
[ 33%] Building CXX object CMakeFiles/obs-backgroundremoval.dir/src/background-filter.cpp.o
[ 66%] Building CXX object CMakeFiles/obs-backgroundremoval.dir/src/plugin-main.cpp.o
make[2]: *** No rule to make target `<path>/obs-studio/build/libobs/libobs.dylib', needed by `obs-backgroundremoval.so'.  Stop.
make[1]: *** [CMakeFiles/obs-backgroundremoval.dir/all] Error 2
make: *** [all] Error 2

• I looked online and saw that other plugins are compiled using make -j4. I executed that and it worked(?):

Scanning dependencies of target obs-backgroundremoval
[ 66%] Building CXX object CMakeFiles/obs-backgroundremoval.dir/src/plugin-main.cpp.o
[ 66%] Building CXX object CMakeFiles/obs-backgroundremoval.dir/src/background-filter.cpp.o
[100%] Linking CXX shared module obs-backgroundremoval.so
[100%] Built target obs-backgroundremoval

However after I copy the plugin, OBS fails to start:

Exception Type:        EXC_BAD_INSTRUCTION (SIGILL)
Exception Codes:       0x0000000000000001, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Illegal instruction: 4
Termination Reason:    Namespace SIGNAL, Code 0x4
Terminating Process:   exc handler [19910]

Application Specific Information:
detected buffer overflow

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_c.dylib             	0x00007fff67a3dfca __chk_fail_overflow.cold.1 + 16
1   libsystem_c.dylib             	0x00007fff67a3b214 __chk_fail_overflow + 9
2   libsystem_c.dylib             	0x00007fff67a3b848 __memcpy_chk + 18
3   libobs.0.dylib                	0x0000000110daf06b obs_register_source_s + 299
4   obs-backgroundremoval.so      	0x0000000028444925 obs_module_load + 21
5   libobs.0.dylib                	0x0000000110dade0e obs_init_module + 78
6   libobs.0.dylib                	0x0000000110daec80 load_all_callback + 64
7   libobs.0.dylib                	0x0000000110daeb7e obs_find_modules + 2046
8   libobs.0.dylib                	0x0000000110dae363 obs_load_all_modules + 35
9   com.obsproject.obs-studio     	0x000000010e0d47ce OBSBasic::OBSInit() + 734
10  com.obsproject.obs-studio     	0x000000010e0b0cd9 OBSApp::OBSInit() + 681
11  com.obsproject.obs-studio     	0x000000010e0b4d28 main + 5032
12  libdyld.dylib                 	0x00007fff6796ecc9 start + 1

And that's not ideal. Please advise.

[royshil]

first i suggest not to supply LIBOBS_LIB and LIBOBS_INCLUDE_DIR directly, rather to stick to obsPath, obsLibPath and obsIncludePath
and let CMake find the file it needs.
if that doesn't work, check your paths again.

but it seems you got CMake happy at some point which is good.

for the crash, it might be that it can't find the model file or the onnxruntime dependency.

can you run OBS from the command line to capture the runtime logs?

[royshil]

I've added scripts to build opencv and onnxruntime as static libraries.
this removes the dependency on the homebrew dynamic libraries and encapsulate everything in the plugin .so file.
it should be more robust.
please give it a try

[mctrafik]

Hey Roy.

Thanks for the super quick turnaround. I tried the new instructions. I think I'm having issues building.

I had to do brew install wget (minor thing to add to instructions probably).

When installing opencv, the only thing that popped at me was:

-- Performing Test HAVE_LINK_AS_NEEDED
-- Performing Test HAVE_LINK_AS_NEEDED - Failed

THere were many more failures when compiling onnx:

-- Performing Test HAS_UNUSED_BUT_SET_VARIABLE
-- Performing Test HAS_UNUSED_BUT_SET_VARIABLE - Failed
-- Performing Test HAS_UNUSED_PARAMETER
-- Performing Test HAS_UNUSED_PARAMETER - Success
-- Performing Test HAS_UNUSED_VARIABLE
-- Performing Test HAS_UNUSED_VARIABLE - Success
-- Performing Test HAS_CAST_FUNCTION_TYPE
-- Performing Test HAS_CAST_FUNCTION_TYPE - Failed
-- Performing Test HAS_PARENTHESES
-- Performing Test HAS_PARENTHESES - Success
-- Performing Test HAS_USELESS_CAST
-- Performing Test HAS_USELESS_CAST - Failed
-- Performing Test HAS_NONNULL_COMPARE
-- Performing Test HAS_NONNULL_COMPARE - Failed
-- Performing Test HAS_TAUTOLOGICAL_POINTER_COMPARE
-- Performing Test HAS_TAUTOLOGICAL_POINTER_COMPARE - Success
-- Performing Test HAS_CATCH_VALUE
-- Performing Test HAS_CATCH_VALUE - Failed
-- Performing Test HAS_MISSING_BRACES
-- Performing Test HAS_MISSING_BRACES - Success
-- Performing Test HAS_IGNORED_ATTRIBUTES
-- Performing Test HAS_IGNORED_ATTRIBUTES - Success
-- Performing Test HAS_DEPRECATED_COPY
-- Performing Test HAS_DEPRECATED_COPY - Failed
-- Performing Test HAS_DEPRECATED_DECLARATIONS
-- Performing Test HAS_DEPRECATED_DECLARATIONS - Success
-- Performing Test HAS_CLASS_MEMACCESS
-- Performing Test HAS_CLASS_MEMACCESS - Failed
-- Performing Test HAS_MAYBE_UNINITIALIZED
-- Performing Test HAS_MAYBE_UNINITIALIZED - Failed

I have no clue if these are warning or failures.

Either way, when I try to build now, I get this error:

% cmake .. \
-DobsLibPath=/Applications/OBS.app/Contents/Frameworks \
-DobsIncludePath="../../obs-studio/libobs" \
-DLIBOBS_INCLUDE_DIR="../../obs-studio/libobs" \
-DLIBOBS_LIB="../../obs-studio/build/libobs/libobs.dylib"
-- Onnxruntime_INCLUDE_DIRS /path/obs-backgroundremoval/build/onnxruntime//include;/path/obs-backgroundremoval/build/onnxruntime//include/onnxruntime/core/session
CMake Error at build/opencv/lib/cmake/opencv4/OpenCVModules.cmake:111 (message):
  The imported target "ade" references the file

     "/path/obs-backgroundremoval/build/opencv/lib/opencv4/3rdparty/libade.a"

  but this file does not exist.  Possible reasons include:

  * The file was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and contained

     "/path/obs-backgroundremoval/build/opencv/lib/cmake/opencv4/OpenCVModules.cmake"

  but not all the files it references.

Call Stack (most recent call first):
  build/opencv/lib/cmake/opencv4/OpenCVConfig.cmake:133 (include)
  CMakeLists.txt:97 (find_package)


-- Configuring incomplete, errors occurred!

I'm not sure what that's all about.

[royshil]

i added a flag to make opencv not use libade
please try building opencv again

those errors you see in cmake are fine.

[mctrafik]

Cool!

I can run cmake .. now. The thing that I thought looked concerning was when running cmake --build I saw this:

ld: warning: direct access in function 'onnx::AttributeProto::_InternalParse(char const*, google::protobuf::internal::ParseContext*)' from file 'onnxruntime//lib/libonnx_proto.a(onnx-ml.pb.cc.o)' to global weak symbol 'void google::protobuf::internal::arena_destruct_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(void*)' from file 'onnxruntime//lib/libonnxruntime_graph.a(graph.cc.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
ld: warning: direct access in function 'onnx::NodeProto::_InternalParse(char const*, google::protobuf::internal::ParseContext*)' from file 'onnxruntime//lib/libonnx_proto.a(onnx-ml.pb.cc.o)' to global weak symbol 'void google::protobuf::internal::arena_destruct_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(void*)' from file 'onnxruntime//lib/libonnxruntime_graph.a(graph.cc.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
ld: warning: direct access in function 'onnx::NodeProto::_InternalParse(char const*, google::protobuf::internal::ParseContext*)' from file 'onnxruntime//lib/libonnx_proto.a(onnx-ml.pb.cc.o)' to global weak symbol 'void google::protobuf::internal::arena_destruct_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(void*)' from file 'onnxruntime//lib/libonnxruntime_graph.a(graph.cc.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
ld: warning: direct access in function 'onnx::TensorProto::_InternalParse(char const*, google::protobuf::internal::ParseContext*)' from file 'onnxruntime//lib/libonnx_proto.a(onnx-ml.pb.cc.o)' to global weak symbol 'void google::protobuf::internal::arena_destruct_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(void*)' from file 'onnxruntime//lib/libonnxruntime_graph.a(graph.cc.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
ld: warning: direct access in function 'google::protobuf::internal::InternalMetadataWithArenaBase<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, google::protobuf::internal::InternalMetadataWithArenaLite>::mutable_unknown_fields_slow()' from file 'onnxruntime//lib/libonnx_proto.a(onnx-ml.pb.cc.o)' to global weak symbol 'typeinfo for google::protobuf::internal::InternalMetadataWithArenaBase<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, google::protobuf::internal::InternalMetadataWithArenaLite>::Container' from file 'onnxruntime//lib/libprotobuf-lite.a(parse_context.cc.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
ld: warning: direct access in function 'google::protobuf::internal::InternalMetadataWithArenaBase<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, google::protobuf::internal::InternalMetadataWithArenaLite>::mutable_unknown_fields_slow()' from file 'onnxruntime//lib/libonnx_proto.a(onnx-ml.pb.cc.o)' to global weak symbol 'void google::protobuf::internal::arena_destruct_object<google::protobuf::internal::InternalMetadataWithArenaBase<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, google::protobuf::internal::InternalMetadataWithArenaLite>::Container>(void*)' from file 'onnxruntime//lib/libprotobuf-lite.a(parse_context.cc.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
ld: warning: direct access in function 'void google::protobuf::internal::RepeatedPtrFieldBase::MergeFrom<google::protobuf::RepeatedPtrField<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >::TypeHandler>(google::protobuf::internal::RepeatedPtrFieldBase const&)' from file 'onnxruntime//lib/libonnx_proto.a(onnx-ml.pb.cc.o)' to global weak symbol 'void google::protobuf::internal::arena_destruct_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(void*)' from file 'onnxruntime//lib/libonnxruntime_graph.a(graph.cc.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
ld: warning: direct access in function 'onnx::FunctionProto::_InternalParse(char const*, google::protobuf::internal::ParseContext*)' from file 'onnxruntime//lib/libonnx_proto.a(onnx-operators-ml.pb.cc.o)' to global weak symbol 'void google::protobuf::internal::arena_destruct_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(void*)' from file 'onnxruntime//lib/libonnxruntime_graph.a(graph.cc.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
ld: warning: direct access in function 'onnx::FunctionProto::_InternalParse(char const*, google::protobuf::internal::ParseContext*)' from file 'onnxruntime//lib/libonnx_proto.a(onnx-operators-ml.pb.cc.o)' to global weak symbol 'void google::protobuf::internal::arena_destruct_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(void*)' from file 'onnxruntime//lib/libonnxruntime_graph.a(graph.cc.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
ld: warning: direct access in function 'onnx::FunctionProto::_InternalParse(char const*, google::protobuf::internal::ParseContext*)' from file 'onnxruntime//lib/libonnx_proto.a(onnx-operators-ml.pb.cc.o)' to global weak symbol 'void google::protobuf::internal::arena_destruct_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(void*)' from file 'onnxruntime//lib/libonnxruntime_graph.a(graph.cc.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
ld: warning: direct access in function 'onnx::MapProto::_InternalParse(char const*, google::protobuf::internal::ParseContext*)' from file 'onnxruntime//lib/libonnx_proto.a(onnx-data.pb.cc.o)' to global weak symbol 'void google::protobuf::internal::arena_destruct_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(void*)' from file 'onnxruntime//lib/libonnxruntime_graph.a(graph.cc.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
[100%] Built target obs-backgroundremoval

But otherwise I think it succeeded to build (it did spit out the *.so) file.

However when running OBS it just exits saying there was an illegal instruction. Last time it popped up a window w/ debug info. It doesn't do that anymore.

I don't think this is useful for debugging, bug included here for completeness:

% /Applications/OBS.app/Contents/MacOS/obs
info: CPU Name: Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
info: CPU Speed: 2400MHz
info: Physical Cores: 8, Logical Cores: 16
info: Physical Memory: 65536MB Total
info: OS Name: Mac OS X (NSMACHOperatingSystem)
info: OS Version: Version 10.15.7 (Build 19H1030)
info: Kernel Version: 19.6.0
error: hotkeys-cocoa: Getting keyboard keys failed
error: hotkeys-cocoa: Getting keyboard keys failed
error: hotkeys-cocoa: Getting keyboard keys failed
error: hotkeys-cocoa: Getting keyboard keys failed
error: hotkeys-cocoa: Getting keyboard keys failed
info: hotkeys-cocoa: Using layout 'com.apple.keylayout.Colemak'
info: Current Date/Time: 2021-04-29, 21:00:45
info: Browser Hardware Acceleration: true
info: Portable mode: false
QMetaObject::connectSlotsByName: No matching signal for on_advAudioProps_clicked()
QMetaObject::connectSlotsByName: No matching signal for on_advAudioProps_destroyed()
QMetaObject::connectSlotsByName: No matching signal for on_actionGridMode_triggered()
QMetaObject::connectSlotsByName: No matching signal for on_program_customContextMenuRequested(QPoint)
QMetaObject::connectSlotsByName: No matching signal for on_transitionRemove_clicked()
info: OBS 26.1.2 (mac)
info: ---------------------------------
info: ---------------------------------
info: audio settings reset:
	samples per sec: 48000
	speakers:        2
info: ---------------------------------
info: Initializing OpenGL...
info: Loading up OpenGL on adapter ATI Technologies Inc. AMD Radeon Pro 5500M OpenGL Engine
info: OpenGL loaded successfully, version 4.1 ATI-3.10.19, shading language 4.10
info: ---------------------------------
info: video settings reset:
	base resolution:   1280x720
	output resolution: 1280x720
	downscale filter:  Bicubic
	fps:               30/1
	format:            NV12
	YUV mode:          709/Partial
info: NV12 texture support not available
info: Audio monitoring device:
	name: Default
	id: default
info: ---------------------------------
warning: Failed to load 'en-US' text for module: 'decklink-captions.so'
warning: Failed to load 'en-US' text for module: 'decklink-ouput-ui.so'
2021-04-29 21:00:46.154 obs[7659:1144045] mac-virtualcam(DAL): PlugInMain version=1.3.0
2021-04-29 21:00:46.154 obs[7659:1144045] mac-virtualcam(DAL): HardwarePlugIn_QueryInterface 
2021-04-29 21:00:46.154 obs[7659:1144045] mac-virtualcam(DAL): HardwarePlugIn_Release sRefCount now = 0
2021-04-29 21:00:46.155 obs[7659:1144045] mac-virtualcam(DAL): HardwarePlugIn_InitializeWithObjectID self=0x20f08478
2021-04-29 21:00:46.155 obs[7659:1144045] mac-virtualcam(DAL): HardwarePlugIn_ObjectSetPropertyData OBSDALDevice(36) kCMIOObjectPropertyListenerAdded self=0x20f08478 data(int)=1684629094
2021-04-29 21:00:46.155 obs[7659:1144045] mac-virtualcam(DAL): HardwarePlugIn_ObjectSetPropertyData OBSDALDevice(36) kCMIOObjectPropertyListenerAdded self=0x20f08478 data(int)=1869180523
2021-04-29 21:00:46.155 obs[7659:1144045] mac-virtualcam(DAL): HardwarePlugIn_ObjectSetPropertyData OBSDALDevice(36) kCMIOObjectPropertyListenerAdded self=0x20f08478 data(int)=1885762592
warning: A DeckLink iterator could not be created.  The DeckLink drivers may not be installed
info: No blackmagic support
info: [mac-virtualcam] version=1.3.0
info: [VideoToolbox encoder]: Adding VideoToolbox H264 encoders
warning: Failed to load 'en-US' text for module: 'obs-backgroundremoval.so'
zsh: illegal hardware instruction  /Applications/OBS.app/Contents/MacOS/obs

[mctrafik]

Can you static link to the libs and upload another version of the *.so file?

[royshil]

@mctrafik can you make sure your OBS repo checkout (for libobs includes) is at the 26.1.2 branch? (before you rebuild the plugin)

this was the solution here #4

I can add the static .so - but you're almost there yourself!

[mctrafik]

Woo! Got it to work.

I use GHE, so had to run git clone --single-branch -b 26.1.2 https://github.com/obsproject/obs-studio.git. Thanks for the tip.

Specifying DobsLibPath and DobsIncludePath still didn't work for me. 🤷🏻 Had to point directly to the framework lib:

cmake .. \
    -DQTDIR=/usr/local/opt/qt \
    -DLIBOBS_INCLUDE_DIR="../../obs-studio/libobs" \
    -DLIBOBS_LIB="/Applications/OBS.app/Contents/Frameworks/libobs.0.dylib"

Then it started!

My only issue now is that doesn't recognize my face:

It doesn't think my hair and forehead are part of my face. --.--

[royshil]

@mctrafik can we close the issue?

[mctrafik]

Of course. Thank you for your help!