Windows: can't import videos via pix_film/movie

[ghost]

I'm a pd user with Windows 10 and I'm experiencing some several situations in several circumstances.

first of all, when I tried to import video via [pix_film]-[pix_texture] or [pix_movie], it suddenly freezes while turning on with [auto $1( and everything doesn't work (load the video was successful and it was an avi file).
a last message was

[pix_film]: loaded file: (directory-video file name) with 35 frames (720x576) at 25.000000 fps

I believe it's not just a problem with my file because I experienced same problem with a demo movie file in pix_film-help, or pix_movie-help.

this happened on Pd-Vanilla. and even Pd-L2Ork(a.k.a.Purr data) doesn't load the video (unable to load~~). meanwhile everything goes fine on Pd-extended but because this is already a dead product, I want to continue my project on Pd-Vanilla or L2Ork. what could be the causes and what should I do in this situation? thank you in advance

[umlaeute]

thanks for the bug-report.

• which version of Pd is this? which architecture (32bit, 64bit)?
• which version of Gem are you using? how did you install it?

can you start Pd from the cmdline, and see whether it prints something to the console?

something like:

C:\Program Files\Pd\bin\pd.com -stderr -lib Gem

[jamshark70]

I'm having a similar problem -- different feedback from the software, but the issue title "can't import videos" applies.

Pd 0.49.0 (64 bit)
Gem 0.94 (git v0.94) installed by deken

[bang button] --> [openpanel] --> [open $1( --> [pix_film]

In the file dialog, I chose "E:/2018-19-spring/apres-la-pluie-silent.mp4"

[GEM:imageTIFF] E:/2018-19-spring/apres-la-pluie-silent.mp4 Not a TIFF or MDI file, bad magic number 0 (0x0)
[pix_film]: unable to open file E:/2018-19-spring/apres-la-pluie-silent.mp4

Running from the command line with -stderr shows that it is in fact going through [GEM:imageMAGICK] despite the fact that it is a video file.

I need to run to a meeting now, and I don't have regular access to a Windows machine, so it will be next week sometime before I can try anything else.

[umlaeute]

Gem will fallback to opening videos as images (so you can load a JPG with [pix_film]); obviously the image-loader cannot really open an MP4.

however, i think this is indeed a different issue than the one @shy8500 is having (for them, it seems that the file can be opened (as it reports frame size and number), but then "something" goes wrong). so @jamshark70, please report your problems as another issue.

[jamshark70]

Ok, that makes sense, it's fine to disregard my message.

I found some sort of open source DirectShow plugin set which might handle h264 video but ran out of time to try it. If that fails, I'll open a new issue.

Thanks for the quick feedback!

[ghost]

hello. I'm using Pd version 0.49.0 (64 bit) and downloaded GEM via "find externals"-deken archive (version 0.94).

my console prints like below:

GEM: Graphics Environment for Multimedia
verbose(2): GEM: ver: 0.94.git v0.94
verbose(2): GEM: compiled  on Feb 12 2019
verbose(2): GEM: maintained by IOhannes m zmoelnig
verbose(2): GEM: Authors :      Mark Danks (original version)
verbose(2): GEM:                Chris Clepper
verbose(2): GEM:                Cyrille Henry
verbose(2): GEM:                IOhannes m zmoelnig
verbose(2): GEM: with help by Guenter Geiger, Daniel Heckenberg, James Tittle, Hans-Christoph Steiner, et al.
verbose(2): GEM: found a bug? miss a feature? please report it:
verbose(2): GEM:        homepage https://gem.iem.at/
verbose(2): GEM:        bug-tracker https://bugs.gem.iem.at/
verbose(2): GEM:        mailing-list https://lists.puredata.info/listinfo/gem-dev/
open: C:\Program Files\Common Files\Pd/gem.conf: No such file or directory
open: C:\Users\winds\AppData\Roaming\Pd/gem.conf: No such file or directory
open: ./gem.conf: No such file or directory
verbose(2): GEM: compiled for MMX/SSE2 architecture
verbose(2): GEM: using SSE2 optimization
verbose(2): GEM: detected 4 CPUs
load plugins 'film' in 'C:/Users/winds/Documents/Pd/externals/Gem/'
pattern : C:/Users/winds/Documents/Pd/externals/Gem/gem_film*.dll
dylib loading file 'C:/Users/winds/Documents/Pd/externals/Gem/gem_filmAVI.dll'!
GemDylib failed: #126: GEM: image loading plugins: magick SGI STB jpeg tiff
GEM: film loading plugins: DirectShow
GEM: image saving plugins: SGI STB jpeg magick tiff
GEM: model loading plugins: ASSIMP3 OBJ

***** VIDEOINPUT LIBRARY - 0.2000 - TFW2013 *****

GEM: video capture plugins: VIDS decklink vnc

[umlaeute]

@shy8500 thanks for the additional info.
so what is the actual problem you are facing (please be a bit more specific than "everything doesn't work")

[ghost]

it literally "doesn't work". entire program freezes and everything I can do with Pd - such as clicking bang or toggle, dragging objects, keyboard shortcuts, etc... even closing windows doesn't work literally I can't do anything as soon as I try to play the video file with [auto $1(. but fortunately dragging, max/minimizing pd window are OK. I have to close whole Pd by force.

[umlaeute]

so does it only freeze if you use [auto 1(?
so what do you get if you don't use [auto 1(?
what if you t instead just directly seek to a single frame (sending e.g. 10 to the 2nd inlet of [pix_film])?

also: what do you get during the freeze in the cmd-console?
do you get more if you raise verbosity by passing -verbose -verbose (the more the merrier) as flag when starting Pd?

[ghost]

same result with sending a single frame. is there still something else I could do instead? while freezeng, no messages on the console. here is the messages until freezes

matchbox: OSC-pattern matching code (c) Matt Wright, CNMAT
verbose(2): comport - PD external for unix/windows
LGPL 1998-2015, Winfried Ritsch and others (see LICENSE.txt)
Institute for Electronic Music - Graz
verbose(3): [pix_film]: DirectShow support
verbose(3): [pix_film]: image support
[pix_film]: loaded file: C:/Users/winds/Documents/Pd/externals/Gem/examples/data/alea.mpg with 162 frames (320x240) at 30.000030 fps

• I'm sorry but I couldn't get what is "raise verbosity". what should I do to test that?

[kubriel]

you run your puredata from commandline with command like 'pd -lib Gem -verbose -verbose'

[agraef]

FWIW, I can confirm @shy8500's problem on Windows 10 with the DS backend, using Pd 0.49 64 bit and the latest 64 bit Gem package from Deken. It's easy to reproduce, load the 05.film.pd example from Gem/examples/04.pix, in the patch open the alea.mpg video, push [create( to create the Gem window, and finally click on the toggle of the [auto $1] message => Pd locks up completely and has to be killed from the Windows task manager. Happens every time. (@umlaeute This sounds an awful lot like the lockups I had with QT4L on Linux, cf. #233, maybe there's a similar one-off bug playing out there?)

I'm also experiencing similar issues with the DS backend as @jamshark70, i.e., Gem's DS backend just refuses to load most video files in present-day formats like MPEG4, even though they work fine in Windows Media Player or VLC.

Here's what I get with the -verbose option, in vanilla (0.49.0 64 bit, using the Gem 0.94 64 bit package dated March 2019 from Deken)::

[pix_film]: DirectShow support
[pix_film]: image support
[GEM:videoDS] Error occured while playing or pausing or opening the file
[GEM:imageMAGICK] reading problem[pd.com]: NoDecodeDelegateForThisImageFormat `AVI' @ error/constitute.c/ReadImage/512
[GEM:imageTIFF] C:/Users/ag/Desktop/Test/gem-test/Gem/alea.avi Not a TIFF or MDI file, bad magic number 0 (0x0)
[pix_film]: unable to open file: alea.avi

So it appears that the filmDS and videoDS plugins did load successfully, but for some reason they either can't load the video or they can't play them back. This uses an AVI video, but the same happens with MPEG4. (The same also happens with the latest Purr Data and the latest Gem from git compiled from source, so it's not just an issue with the binaries IMHO.)

Here is the example that I used for testing (includes the 05film.pd patch and the alea.avi video):
05film.zip. That example works just fine on Linux (using QT4L).

I think that the only way I could ever get video playback in Gem to work on Windows was with @avilleret's experimental 32 bit builds over at https://github.com/avilleret/Gem, via the QT backend. But those packages have their kinks, too (either QT or VIDS missing; QT auto plays back at the wrong speed).

I should also add that in either case, video capture from the webcam via the VIDS plugin works fine.

So that's what I know about the present state of Gem video on Windows. :) I think that getting the DS backend to work properly is definitely the way to go, QT being 32 bit only (also, AFAICT QT needs to be built using MSVC, and I prefer to build stuff with msys2/mingw).

[umlaeute]

i haven't tested now, but I'm pretty sure that I did some successful tests of opening footage files on Windows.

Pro-Tip: 👩‍🏫 Make sure you have the K-Lite Code Pack installed.

[jamshark70]

Huh. K-Lite Codecs actually do work on my Win10 system. Thanks for the tip!

It would be immensely helpful to include this in the GEM documentation, to avoid people making claims such as "it doesn't work at all in Windows."

[umlaeute]

@jamshark70 thanks for confirming.

While I did install the codec-pack during testing (mostly because the homer.avi uses a codec that has been removed from Windows in the last decades or so), I was not sure whether this would actually solve the issue.

[ghost]

thanks for the tip, but it still troubles after installing K-Lite Codec (Standard version) for me. do I install with specific settings? (increasing verbosity didn't work well either)

[ghost]

@umlaeute hey, I believe I figured out what was a problem.
I have tested pix_film and pix_movie via alea.mpg (demo video which could load directly in pix_movie.help patch). I found another video sample file homer.avi in GEM data folder and replaced it and it finally worked. also I had tried to convert alea.mpg to avi file and tried it to play and it worked too! I still don't know why MPG video freezes the program even though I already installed K-LITE CODEC but at least I finally concluded I can move my GEM patch with AVI file and MPG file was the problem. (still though I'd be appreciate someone could explain what was a problem with MPG file exactly). to some people with same problem, I'd like to recommend to test with various video format and codecs. (but at least I believe we must have K-LITE CODEC as a base. thank you for mr. umlaute, for letting me know this CODEC!)

[umlaeute]

@shy8500 how about closing this issue (without re-opening it) then?

[ghost]

OK. just not got used to using github system yet. thanks!

[jamshark70]

FWIW, on this issue, I got stuck in the middle between the suggestion here to use K-Lite codecs, vs the Purr-Data maintainers' refusal even to recommend non-free dependencies "at your own risk" and their insistence that GEM should be able to find and use VLC codecs (it doesn't).

So I don't know who is right and it was a somewhat awkward position to be in as a user (developer A: "it's fixed by installing xyz"; developer B: "we will never recommend xyz").

I don't have that much stake in it except to make the issue's status, and way forward, easily visible to incoming users. Visibility matters because I went for months thinking video was totally broken in Windows. Users have to be able to find this information in a readme or something, or they will reach the same conclusion I did.

[umlaeute]

regarding documentation: please submit a PR to fix the documentation - preferably in a way that reaches the target audience (Windows users) but doesn't irritate the rest of the world.

regarding VLC: the current VLC-backend in Gem is for [pix_video] only. while it does allow you to play back video files, it doesn't give you the frame-accurate interface of [pix_film] (seek frame), but rather a streaming oriented interface (start playback, stop playback). which might be what you want. or not.

<rant>

regarding your awkward position: the folks of Purr Data are just some "random people" and not "developer B".

don't get me wrong: they are certainly doing great work, and doing a lot of fantastic development.
i also value their position regarding free software (to refuse recommending non-free software).

but they are not Gem developers. they just bundle Gem with Purr Data. so i guess the situtation is similar to getting some broken product from amazon: who is to blame: the producer, amazon or the postman? i don't know but it doesn't help complaining that the producer, amazon and the postman all give you different advise/excuses/...

</rant>

[jamshark70]

regarding documentation: please submit a PR to fix the documentation - preferably in a way that reaches the target audience (Windows users) but doesn't irritate the rest of the world.

OK, fair.

regarding VLC: the current VLC-backend in Gem is for [pix_video] only. while it does allow you to play back video files,

I wouldn't have guessed that.

it doesn't give you the frame-accurate interface of [pix_film] (seek frame), but rather a streaming oriented interface (start playback, stop playback). which might be what you want. or not.

Yeah, I kinda need to be able to glitch it up.

Re: rant -- I suppose the way forward is -- if Gem documents that video objects should in most cases work in Windows 10 by using these nonfree codecs, and if Purr Data devs wish to bundle Gem, then: either they bundle it as is (including the documentation), or they modify the documentation (and then user complaints are their responsibility to deal with), or they don't bundle it.

[umlaeute]

I wouldn't have guessed that.

but why?
afaik, the only documentation hint of the possibility to use libVLC is in the context of [pix_video].
or was it because someone just mentioned "VLC" and the obvious conclusion was "film" (because that's what VLC usually stands for)

[umlaeute]

and btw: i dont think the FLOSS "zeal" would extend to actively removing documentation.

[jamshark70]

but why?

I didn't see anything in the pix_video help patch about opening video files, so I assumed that it had no such feature. I might have overlooked it, but after your message, I did look at the help patch again and I just don't see it.

was it because someone just mentioned "VLC" and the obvious conclusion was "film" (because that's what VLC usually stands for)

This is exactly what I meant by "awkward position." I shouldn't be in the middle between you and the Purr Data team. I do understand that you're not under any particular obligation to seek them out, but I'm not interested in acting as anybody's representative here except my own.

They had a question why pix_film couldn't use free codecs bundled with VLC. Why they thought so, I have no idea.

At this point, my plan is to submit a documentation PR and then notify the Purr Data folks of the findings here. What they do with that is up to them, entirely.