Trouble getting Gstreamer 1.0 to work properly on OSX 10.9.4 with MacPorts

[SenshiSentou]

Hello!

I posted about this issue earlier, but the response was, well, non-existent. I'm guessing this is more of a support-thing rather than a bug report, but filing it here too, just in case!

A synopsis of my situation:

• I installed pygobject3, gstreamer and gst-plugins-[base | good | bad | ugly] using MacPorts
• I can import gi just fine from both a Python process, and a Kivy one
• I can even get Gst.version() from within my Kivy app
• My video provider keeps returning null

I found that if I uncommented line 210 in kivy/core/video/init.py (video_providers += [('gi', 'video_gi', 'VideoGi')]) I actually get gstplayer as my video provider; huzzah!

Then though, it starts complaining about a segfault in Resources/lib/gst-plugins/___so. Removing that file (as suggested by the error message) just shifts the error to the next one, so I just emptied the entire folder. \0/

Ok, so now the app starts, and we don't get any errors (just a few warnings RE SDL_xxx classes being implemented in both Kivy's and SDL's libSDL-1.2.0.dylib). However, my video files still don't load. I've tried binding my test callback to both loaded and on_load, but no dice. Does anyone have any ideas on how to proceed? Video support is rather crucial in this specific app, so it would be great to see this resolved. =)

Thanks,
Patrick

[SenshiSentou]

Small update: Replacing the contents of Resources/lib/gst-plugins/ with the .so files from my MacPorts install (/opt/local/lib/gstreamer-1.0/) fixes the segfaults, but I still can't load any video (tried with custom .mpg and .avi, as well as included softboy.avi).

[tito]

Hi,

I guess our FAQ is misleading. The gi is not used, and deactivated for a purpose:
http://kivy.org/docs/faq.html?highlight=gstreamer#gstreamer-compatibility

There is no way to know which version you have installed prior the import. And as soon as we imported Gi, there is no going back. And it's not crossplatform yet.

The latest decision was to create our own gstreamer player, which is available now. If you have any issues with it, then try first to activate the gstreamer debug, upload, and let us see what's going on for you on the default version we ship (not the version you changed, which can be a MESS if you don't fix rpath in the .so, shared deps that are copied, etc. The portable installation is not the same than the .so/.dylib you can find on your computer).

So to start a video that fail:

GST_DEBUG=*:5 python main.py

If you use the kivy wrapper, edit it and add the export GST_DEBUG=*:5 before

[SenshiSentou]

Argh, it seems I broke being able to just use python main.py. I'm assuming you wanted me to put the following in Kivy's script file?

exec $(GST_DEBUG=*:5 python -c "import os, sys; print os.path.normpath(sys.prefix)")/bin/python2.7 "$@"

The log is exactly the same either way =/ http://www.pasteall.org/53801

EDIT: I found it weird using the python command didn't work anymore, so I did a clean install of Kivy, which also guarantees all .sos are back to their original state.

[tito]

The paste doesn't contains any gstreamer debug :( Can you try GST_DEBUG=*:5 gst-inspect ?

[SenshiSentou]

Yeah, I can't get anything using just GST_DEBUG=*:5 python .... =/ gst-inspect isn't a valid command, but calling GST_DEBUG=*:5 gst-inspect-1.0 > output.txt Gives me two things.

• A nice, clean output inside the .txt: http://www.pasteall.org/53805
• A horrendous-looking metric tonne of output in my console window[1].

[1.1] Formatted .rtf
[1.2] Plain text .txt

[SenshiSentou]

Hey Tito,

Did you happen to get a moment to take a look at this by any chance? =)

[SenshiSentou]

Interesting. When modifying Kivy's script file's export GST_REGISTRY_FORK to "yes" I get some different console output: http://www.pasteall.org/53948/bash

No idea if this is useful info, but figured I'd post it here.

[tito]

It's bad. There is a mix between the plugins / library we ship within Kivy, and the one in your system:

0:00:00.000285000 82943 0x7fe60bd12a00 INFO                GST_INIT gst.c:500:gboolean init_pre(GOptionContext *, GOptionGroup *, gpointer, GError **): Using library installed in /opt/local/lib

It should never do that.

A: did you ever manually replaced files from your system/gstreamer and the one in the portable package?
B: can you share the output of your env within the kivy script, and outside the kivy script?
C: i just understood. You launched the system gst-inspect. It's not useful. Yes you have a correct gstreamer installation. But this one should not be used if you use the portable package, because the portable have already gstreamer. And for the sake of the next 1.9 portable package, it would be nice to see what's wrong for you :) So could you edit the kivy script and invoke env GST_DEBUG=*:5 python main.py ?

[SenshiSentou]

Argh, I didn't of that - how stupid. I haven't set up being able to use python main.py; hopefully using kivy gives the same result?

Plain-text paste: http://www.pasteall.org/54144/bash
.rtf with original shell coloring for ease of reading: https://dl.dropboxusercontent.com/u/971175/KivyGST.rtf

[tito]

Same yep: 0:00:00.000140000 9005 0x7fbef640ea00 INFO GST_INIT gst.c:500:gboolean init_pre(GOptionContext , GOptionGroup *, gpointer, GError *): Using library installed in /opt/local/lib

So why the kivy doesn't use its own library?

[tito]

I just see that we don't set the GST_REGISTRY path in OSX while we set it in Windows. So what can happen is when you use the system gstreamer, it will mess up (or reuse) the kivy gstreamer, because both shared the same registry.

Can you try to add in your kivy script:

export GST_REGISTRY=${SCRIPT_PATH}/registry.bin

?

And ensure this works by checking the line that contains: "reading registry cache:".
The one in your log look like:

0:00:00.002736000  9005 0x7fbef640ea00 INFO            GST_REGISTRY gstregistry.c:1722:gboolean ensure_current_registry(GError **): reading registry cache: /Users/patrickboelens/.cache/gstreamer-1.0/registry.x86_64.bin

[SenshiSentou]

Adding the export does seem to change the registry path to Kivy's:

0:00:00.077046000 22746 0x7fed1eba3000 INFO GST_REGISTRY gstregistry.c:1722:gboolean ensure_current_registry(GError **): reading registry cache: /Applications/Kivy.app/Contents/Resources/registry.bin

But unfortunately it also still seems to be using /opt/local/ files somehow. =/

0:00:00.136343000 22746 0x7fed1eba3000 DEBUG GST_REGISTRY gstregistry.c:1658:GstRegistryScanAndUpdateResult scan_and_update_registry(GstRegistry *, const gchar *, gboolean, GError **): scanning main plugins /opt/local/lib/gstreamer-1.0

[tito]

Can you dump the env (just add env) before starting python within the kivy script?

[SenshiSentou]

Did you mean like so?

exec $(env python -c "import os, sys; print os.path.normpath(sys.prefix)")/bin/python2.7 "$@"

If so, that doesn't give me anything other than the regular Kivy INFO output. Apologies I'm so terrible at this by the way; it's just a little out of my depth. ;)

EDIT: Ah, gotcha! Here's my env: http://www.pasteall.org/54178/bash

[tito]

You have something weird. Your DYLD_FALLBACK_LIBRARY_PATH includes /usr/lib. Did you added it manually in your .profile?

Also, everytime you do a change on the env, for the tests / debugging of this issue, manually remove the registry.bin please.

Did you changed any of the kivy.app contents? (except the kivy script)

[SenshiSentou]

Ah, yes, that was to get PyBonjour to work! Removing it makes my environment look like this: http://www.pasteall.org/54179/bash

I also deleted registry.bin, but no change yet. What is interesting however, is the following warnings I now got right after the regular Kivy INFO block: http://www.pasteall.org/54180/bash

I'm guessing it shouldn't even reach the system SDL here.

[SenshiSentou]

Just started playing with this again and found what seemed to be remnants of earlier testing in the /lib folder. After downloading the official 1.8 distribution and copying over it's lib folder, I got the segfault again. Here's a log of that with GST_DEBUG=*:5 enabled: http://www.pasteall.org/54364/bash

[tito]

There is still a conflict, f*ck!

At the start:
0:00:00.000503000 7497 0x7ffd93ddca00 INFO GST_INIT gst.c:500:gboolean init_pre(GOptionContext *, GOptionGroup *, gpointer, GError **): Using library installed in /opt/local/lib

And then:
0:00:00.007059000 7497 0x7ffd93ddca00 DEBUG GST_REGISTRY gstregistry.c:1344:gboolean gst_registry_scan_path_level(GstRegistryScanContext *, const gchar *, int):<registry0> file /Applications/Kivy.app/Contents/Resources/lib/gst-plugins/libgstaccurip.so not yet in registry

Same issue, mixing system library with the one within the Kivy.app. Different ABI = crash.

[tito]

Ok, ok.

I tried on my computer, guess what, it uses my system library first. At the time of doing the initial OSX package, i have changed all the RPATH to libraries with relative path within the Kivy.app. But the current release explicitely rely on the system (/usr/local/ on my side) one.

I would like to test 2 things with you.

First, could you add in the kivy script:

export DYLD_PRINT_RPATHS=1
export DYLD_PRINT_LIBRARIES_POST_LAUNCH=1
export DYLD_PRINT_LIBRARIES=1

And share the whole output.

Then, add one more line:

export DYLD_LIBRARY_PATH=${SCRIPT_PATH}/lib

And share the whole output too.

Normally, the second one should work as expected, as it explicitly enforce the library to be loaded within the script path, if exists first. So gstreamer-kivy version will be used, and the Gstreamer version printed should be

[INFO              ] [VideoGstplayer] Using Gstreamer 1.2.1.0

But to be sure it works on another computer than mine, i need both log (with the DYLD debug, and without/with DYLD_LIBRARY_PATH).

Thanks!

[SenshiSentou]

Hey, glad to hear you were able to reproduce the problem! I mean, not glad for you/ Kivy, but glad it's not just me messing up. ;)

Unfortunately it still doesn't work though; adding it the DYLD_LIBRARY_PATH makes my video provider become null again. =/

Here are the logs:
Just the DYLD_PRINT_*s: http://www.pasteall.org/54374/bash
With DYLD_LIBRARY_PATH set: http://www.pasteall.org/54375/bash

Note that these are both without the explicit GST_REGISTRY export you mentioned earlier in this thread. I'm not sure if it matters, so here's one more with it set, just for completeness' sake: http://www.pasteall.org/54376/bash

[tito]

Ok, here we are:

dyld: loaded: /Applications/Kivy.app/Contents/Resources/kivy/kivy/lib/gstplayer/_gstplayer.so
dyld: loaded: /Applications/Kivy.app/Contents/Resources/lib/libgstreamer-1.0.0.dylib
dyld: unloaded: /Applications/Kivy.app/Contents/Resources/lib/libgstreamer-1.0.0.dylib
dyld: unloaded: /Applications/Kivy.app/Contents/Resources/kivy/kivy/lib/gstplayer/_gstplayer.so

Now we just need to find out why it doesn't want to load for you :)

Can i get the output of otool -L /Applications/Kivy.app/Contents/Resources/kivy/kivy/lib/gstplayer/_gstplayer.so and otool -L /Applications/Kivy.app/Contents/Resources/lib/libgstreamer-1.0.0.dylib ?

[SenshiSentou]

You sure can! http://www.pasteall.org/54385/bash (yaaay, more local files!)
Thanks so much for all your help! =)

[tito]

Did you copy your own compiled Kivy within the Kivy.app somehow?

[SenshiSentou]

Yes, I'm using a (slightly older) 1.8.1 build (straight git clone from master).

[tito]

Just to be sure, could you try a fresh 1.8.0 version with an example that load a video? To determine if it's about rpath within the compiled kivy version or not. :)

[SenshiSentou]

[INFO ] [VideoGstplayer] Using Gstreamer 1.2.1.0 [INFO ] [Video ] Provider: gstplayer [...] [INFO ] [Base ] Start application main loop [DEBUG ] [VideoGstplayer] Load </Users/patrickboelens/Documents/Project/softboy.avi>

This looks pretty darn promising! The video doesn't actually load though; the callbacks (loaded and on_load are never fired, and the video stays white: http://www.pasteall.org/pic/show.php?id=78141

Here's my example app: http://www.pasteall.org/54392/python (messy, stripped-down code, sorry)

[tito]

Ok, then it's really an issue with rpath, at minimum.

[SenshiSentou]

Hmm, alright! What should I do to proceed? =)

[SenshiSentou]

A-ha! Simply having the video (auto)play fixes that issue. Awesome! Now all that's needed for me is to get it working with version 1.8.1. Can you share what needs to be changed in order for this to work? I'm guessing something in the makefile?

[tito]

It's not something missing in the makefile, it's something missing during the packaging. But i need to be able to fully reproduce it in order to patch it correctly.

[SenshiSentou]

Ah, ok. Please let me know if you need me to test/ log anything then. =)

[SenshiSentou]

Tito, is there any chance there's a make-shift solution I could apply myself in the meanwhile? I'd really like to start using video in my 1.8.1 project. Or if it helps I could attach a small bounty to this issue?

[SenshiSentou]

"The night is dark, and full of terrors" and on this Old Hallow's Eve I'd like to bring your attention to this dreaded bug once more. ;)

Or, in non Halloween-speak: Any progress/ suggestions for me to try yet? =)

[SenshiSentou]

Hey Mathieu,

Any progress on this yet? I hate to keep bumping this, but it's really starting to hold me back right now =/

[SenshiSentou]

Monthly bump

[tito]

Hi @SenshiSentou, i didn't forget you. This month, i worked on a new packaging approach and tools that resolve this.

A new tool, extracted from the build toolchain of gstreamer and repackaged for standalone execution have been pushed: osxrelocator. Have a look at: https://pypi.python.org/pypi/osxrelocator

Second, i tried a complete new approach for the OSX packaging. There is various issue with GStreamer, and mostly pyinstaller after. My goal was to think more as an SDK: you have a Kivy.app on which you can try your software, or use it as a base to make your own app. Then the repo https://github.com/kivy/kivy-osx is born. Internally, it uses official packages for SDL2/_image/_ttf and GStreamer. The end package is bigger, but work and should be more sandboxed. (ie, no mess with the System).

I shared the concept and preliminary packages with the teams, seems to works, but not on all osx version. Contact me directly by email if you wanna test too.

[tito]

More info at https://groups.google.com/forum/#!topic/kivy-dev/rRLeWgLa0LY

[tito]

This is fixed with the new kivy/kivy-osx packaging tools. Next packaged version will contain the fixes :)