Update to GStreamer 1.0

[gohai]

Here is a ZIP file for you to try:

https://github.com/gohai/processing-video/releases/tag/v1.0.2

There's still room for improvement here - an incomplete list:

• The release is currently way bigger because this is including all available GStreamer plugins and related libraries - would be great to find a minimal subset (but e.g. libav seems to be needed to play H.264-encoded videos, and this alone is about 100 MB - unclear)

• I noted too late that the Capture class was just Movie renamed and some additions bolted onto it. Unnecessary functions (e.g. seek()) should be cleaned up, or the current code back-ported onto the original Capture class.

• Device enumeration is currently not available on macOS. That's a limitation with GStreamer.

• Is Capture.list() supposed to return available resolutions also?

• Many system-wide GStreamer-installations on Linux won't be able to play H.264 because of its patent situation. Perhaps it'd be a good idea to rename the Loop example to Basic - so that it is first alphabetically, and hopefully also the first users will try. For this example we could bundle a webm or Ogg Theora encoded file.

• It'd be great to have clear instructions for Linux which packages to install.

[neilcsmith-net]

Just having a quick look at this - couple of comments I'd make.

• Interesting you wrote your own Environment class. However, doesn't Processing already ship with JNA Platform which has this capability already?
• I used to use a version of the LibraryLoader class too, but I'm not sure that it and setting jna.library.path are the right approach. It might be better to use the ENV variable capability to prepend the GStreamer path(s) to the system path, and also set GST_PLUGIN_PATH_1_0 to remove the need to use the Registry? This should cut down code and make it less likely you pick up the wrong version of a dependency (due to the way jna.library.path works).

2c 😄

[gohai]

@neilcsmith
ENV: looked, but didn’t find it
LibraryLoader: I don‘t have any experience with JNA, so perhaps I overlooked something - couldn’t get a simpler (ENV) version to run on Windows, so I hacked up the existing approach by Andres until it launched with no errors.

Care to send patches? Can test them on all platforms easily now

[gohai]

@neilcsmith-net

[neilcsmith-net]

Working code is the most important thing! So, if it's working now I'd merge and think about simplifying later. I'll look a bit more into this as I want to solve it properly for a few other things.

The problem with jna.library.path is that it only sets where the library is loaded from, but not where that library looks for its dependencies - sometimes that's fine, sometimes it isn't. IIRC a Windows DLL for example will use its own location to find dependencies in the same folder, hence why it works for the GStreamer core libs but not the plugins.

I'm currently only setting the ENV variables on Windows using Kernel32 from JNA Platform. There is a libC binding with setenv under the unix package, but looks like it was added in JNA Platform 4.3 (use 4.4+ if you change as 4.3 is incompatible with the GStreamer bindings - their bug!)

[hamoid]

I'm happy to see it finally there! :) I tested all examples on my Linux laptop and they work fine.
Capture.list() is not returning the resolutions though.
The Frames example is not very precise when jumping frames back and forth. Sometimes there seem to be 5 or 6 equal frames. Then if I just go left, right, left, right, left... for a while, sometimes it doesn't do what it should. But if one needs that precision maybe it's better to cache the frames..

Would it make sense to mention somewhere that people can delete the libraries for other OSes to save some space?

Btw: .zip = 270Mb, .7z = 160Mb

[gohai]

@hamoid
Capture.list(): I used the Video library so rarely, I didn't even remember whether it was giving resolutions or not :)
Frames example: the file the example ships with has duplicate frames also, unfortunately. Does this still occur for you when you try a different file? Is this different than with the old version of the library?
Making space: someone just needs to go through and find the most minimal subnet of GStreamer plugins and libraries (see top post) - we simply need to loose some weight.
7z: Not an option, since the library will be distributed through the Contributions Manager and this one doesn't do 7z.

[neilcsmith-net]

@gohai it looks like the old library lists resolutions. You can use the Caps from each Device, loop through all the embedded structures (use caps.size()) and query them for width & height. Note that you'll get a structure for each supported format, so discard duplicate resolutions. And probably good to check the structure has the required fields too - (structure.hasIntField("width") etc.) Tested on Ubuntu and it seems to work correctly.

The Frames example may also not like the play(), jump(), pause() behaviour - you can seek in the paused state if you implement the pre-roll handler as mentioned before.

[gohai]

@neilcsmith-net The play/pause/play didn’t make a difference when I tried removing it. I can only assume it was added for some reason at some point, so I am hesitant to removing it. Let’s first check if the single-frame behavior is indeed different compared to the old lib…

…

On Sun 26. Nov 2017 at 14:25, Neil C Smith ***@***.***> wrote: @gohai <https://github.com/gohai> it looks like the old library lists resolutions. You can use the Caps from each Device, loop through all the embedded structures (use caps.size()) and query them for width & height. Note that you'll get a structure for each supported format, so discard duplicate resolutions. And probably good to check the structure has the required fields too - (structure.hasIntField("width") etc.) Tested on Ubuntu and it seems to work correctly. The Frames example may also not like the play(), jump(), pause() behaviour - you can seek in the paused state if you implement the pre-roll handler as mentioned before. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#84 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEt2K2mUQkM5bb45ODzp3ljYlVdFM-6Hks5s6WbbgaJpZM4Qpvs1> .

[neilcsmith-net]

@gohai I know why it was added - to force a signal to be emitted to update the image - but the pre-roll listener is a better way to do this. By setting it to play and then pause you might end up a frame or two away from where you seek to.

Another cause of this issue is that you're not setting the accurate seek flag. Couple this with an mp4 file and it's not ideal. I'd add SeekFlags.ACCURATE to the seek, and consider using a format like MJPEG for that example (something that is only keyframes anyway).

[codeanticode]

@gohai this is very good work. I think you cover the most obvious improvements in the description of the PR. I would only add that this should become version 2 of the video library, don't you think?

[gohai]

Here are some hints for reducing the size of the library, taken from this FOSDEM presentation:

• strip the debug symbols from the libraries
• run with GST_DEBUG=GST_PLUGIN_LOADING to get a list of used (needed) plugins, and remove the unneeded ones

[neilcsmith-net]

@gohai that second point is interesting, but surely more so at the export sketch level? Is having all 3 platforms for the native libs really needed? What about a wizard that auto-downloads and extracts the right library? Then possibly an export sketch function that only includes the required libs for all platforms as determined by running the sketch? There's some potential for some shared work there.

Was about to say you could probably ditch 32-bit libs as well, but I see that Processing still provides a 32-bit Windows download?! Is that really necessary these days?

Incidentally, the beta seems to have JNA 4.2? That's below the recommended min version.

@codeanticode thanks for the recognition of the immense amount of time that went in upstream to making this work! 😜

[tomjuggler]

Thank you so much for fixing this. Tested working on Manjaro Linux 64bit. I was really worried for a minute there!

Doesn't work with Processing 2.1 though. Guess I need to work out how to compile this with Java 1.6 - some of my sketches still aren't updated to Processing 3.

[yurikleb]

Thanks!!
Works well on Ubuntu 18 and 16 x64
with h264 files.
Saved my day!