Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mpv hangs with mvtools vapoursynth script #1168

Closed
4re opened this issue Oct 10, 2014 · 36 comments
Closed

mpv hangs with mvtools vapoursynth script #1168

4re opened this issue Oct 10, 2014 · 36 comments

Comments

@4re
Copy link

4re commented Oct 10, 2014

Linux x86_64
nvidia-drivers-343.22
mpv 0.6.0
Vapoursynth Core r24
mvtools from git (https://github.com/dubhater/vapoursynth-mvtools)
Script:

import vapoursynth as vs
core = vs.get_core()

src = video_in
clip = core.std.Trim(src, first=0, length=500000)

if clip.format.bits_per_sample != 8:
    clip = core.fmtc.bitdepth(clip, bits=8)

super = core.mv.Super(clip, pel=2)
bv    = core.mv.Analyse(super, isb=True, overlap=4, search=4)
fv    = core.mv.Analyse(super, isb=False, overlap=4, search=4)
clip  = core.mv.FlowInter(clip, super, bv, fv)
clip  = core.std.Interleave([src, clip])

clip.set_output()

I don't have problems using other vapoursynth filters, just this one.
Also no meaningful output on the terminal, it just hangs.

@ghost
Copy link

ghost commented Oct 10, 2014

It hangs because mvtools requests a frame at initialization. Since vf_vapoursynth can deliver frames only after initialization has finished, it's not possible to fix this on mpv's side.

@ghost
Copy link

ghost commented Oct 10, 2014

<nodame> wm4: Well I guess you're out of luck then.

Too bad!

@4re
Copy link
Author

4re commented Oct 11, 2014

It was a beautiful dream while it lasted :(

ghost pushed a commit that referenced this issue Oct 11, 2014
Some VS filters will requests frames from their parent filters while
they're initialized. Thy do this in a blocking manner, and
initialization will not succeed until the frame request is satisfied.
This deadlocked mpv, because we can feed frames to the filter only after
initialization is finished.

Return an error instead of deadlocking.

Note that we (probably) can handle frames being requested during init
fine, as long as the requests don't block initialization. But we can
distinguish this situation, and a simple test seems to indicate VS
usually doesn't do this.

See #1168.
@ghost
Copy link

ghost commented Oct 11, 2014

You still could do something convoluted, like piping vspipe output to mpv.

@ghost
Copy link

ghost commented Oct 11, 2014

Some relevant info from the author of the filters:

<nodame> Filters that take a super clip will want some information that Super passes along in frame properties. Filters that take motion vectors will want some information that Analyse passes along in the frames. <nodame> It's how it's always been done™. <nodame> If arbitrary properties could be attached to nodes, it would not be necessary to request frames during filter creation.

@bjin
Copy link
Contributor

bjin commented Oct 11, 2014

For reference, I met similar issue before (#1026).

@bjin
Copy link
Contributor

bjin commented Oct 11, 2014

@wm4 I think there might be workarounds for this issue as well. From the description from the filter author, Analyse requires some frame properties from output of Super and hence makes a frame request. But for Super, it doesn't necessarily need frame from upstream of mpv filter chain. I think providing dummy video frames for whole vapoursynth filter during filter initialization will probably solve this issue smoothly.

EDIT: I think I made a mistake. The first few output frames of the filter will probably become dummy as well then, but at least the remaining part of video should be fine. It's ugly but better than deadlock or filter failure from my opinion.

@ghost
Copy link

ghost commented Oct 11, 2014

If we feed it dummy frames, we'll also get dummy output.

@bjin
Copy link
Contributor

bjin commented Oct 11, 2014

Yes, I realized that later. But it's only for first few frames.

@ghost
Copy link

ghost commented Oct 11, 2014

It'll also happen after seeks.

@bjin
Copy link
Contributor

bjin commented Oct 11, 2014

It's still a workaround, but this could be avoided with customized vapoursynth script, user can append few dummy frame in front, and seek to real part later.

ghost pushed a commit that referenced this issue Oct 11, 2014
An attempt at fixing #1168.

I see black frames flashing, so it's certainly not perfect.
@ghost
Copy link

ghost commented Oct 11, 2014

There. Doesn't work overly well, but I suppose it's better than failing completely.

@ghost
Copy link

ghost commented Oct 12, 2014

So does this work for you?

@bjin
Copy link
Contributor

bjin commented Oct 12, 2014

It turns out that my laptop is too slow to run this script smoothly(roughly 10fps for a 576p footage), the result looks as expected though(have blank dummy frame in the first ones). I also tried to reduce the blank frames by appending extra frame on the front, but it didn't work.

Anyway I don't use mvtools for realtime playback myself. Although I'm interested in @4re's feedback as well.

@4re
Copy link
Author

4re commented Oct 12, 2014

Sorry for the late reply.

I gave a shot at your recommendations and it works! somewhat.
Sometimes it stutters but the cpu load is not full. I'm not 100% sure this happens because some threads top at some given time as this only happens with 720p or more.
It also is quite sensitive and makes mpv crash if I add more filters or try to seek, but not always.

Other than that it's great to have it running, thank you guys!

@ghost
Copy link

ghost commented Oct 12, 2014

Sometimes it stutters but the cpu load is not full.

Uh, not sure. Maybe try playing with the threading suboptions of vf_vapoursynth.

It also is quite sensitive and makes mpv crash if I add more filters or try to seek, but not always.

This shouldn't happen. Could be a bug in mpv or the filters you use; hard to tell.

@ghedo
Copy link
Member

ghedo commented Oct 13, 2014

I get the "crashes on seek" too. However it doesn't happen on every seek. The error is:

mpv: ../src/core/vsapi.cpp:64: const uint8_t* getReadPtr(const VSFrameRef*, int): Assertion `frame' failed.

i.e. it crashes from inside vapoursynth. May just be a vs bug...

@ghost
Copy link

ghost commented Oct 13, 2014

Actually I'm getting it quite often. MVClipBalls::Update(VSFrameRef const*) crashes in libmvtools.so, probably because mpv returns errors on frame requests when it wants to terminate the filter (like on seek). I suppose it's a bug in the mvtools filter.

@dubhater
Copy link
Contributor

If your source filter calls setFilterError, you're supposed to get a null VSFrameRef* at the other end. I don't think filters are supposed to deal with null VSFrameRef*.

@ghost
Copy link

ghost commented Oct 14, 2014

If your source filter calls setFilterError, you're supposed to get a null VSFrameRef* at the other end. I don't think filters are supposed to deal with null VSFrameRef*.

Not sure if I follow. What else should it return if not a null VSFrameRef*? Though I could make it return dummy frames on EOF. It will probably be a bit slower, but trip up fewer broken filters.

@dubhater
Copy link
Contributor

Your source filter, video_in, does need to call setFilterError and return NULL. This NULL is only supposed to appear at the end of the chain, where you call getFrame/getFrameAsync. It should not appear in any of the filters in-between, because the core is not supposed to execute any more filters after setFilterError was called. At least this is how I hope VapourSynth is meant to work.

@ghost
Copy link

ghost commented Oct 14, 2014

Your source filter, video_in, does need to call setFilterError and return NULL.

It does do that. In the beginning it returned just NULL, but that was fixed some months ago.

Anyway, the vapoursynth dev could reproduce this problem outside of mpv. It seems there is a race in error handling. Using vs.get_core(threads=1) in the script seems to make the crash go away.

@4re
Copy link
Author

4re commented Oct 15, 2014

The crashes are gone after vapoursynth/vapoursynth@8ef3b6c

@ghost
Copy link

ghost commented Oct 15, 2014

I'm still getting deadlocks with that, and I think a follow-up fix is in work.

@ghost
Copy link

ghost commented Oct 15, 2014

I think this should be fully fixed now in vapoursynth git master?

@4re
Copy link
Author

4re commented Oct 16, 2014

For me everything is fixed, thank you.

@4re 4re closed this as completed Oct 16, 2014
@Sperg
Copy link

Sperg commented Nov 2, 2014

Could you PLEASE post the finished script. I've been trying to get frame interpolation happening on GNU/Linux for a week now, this looks to be the beacon of hope I've been searching for. I was just about ready to learn the ffmpeg C library. And this will do frame interpolation right? Right guys?

@Sperg
Copy link

Sperg commented Nov 2, 2014

Just incase that was the finished product I tried your script and got this

┌─[mugi]─[/mpv/mpv-master]
└──╼ ./build/mpv --vf=vapoursynth=/script.vpy --vo=opengl ~/fuck.mp4
Playing: /root/fuck.mp4
[stream] Video (+) --vid=1 () (h264)
[stream] Audio (+) --aid=1 --alang=und (
) (aac)
File tags:
major_brand: mp42
minor_version: 0
compatible_brands: isommp42
creation_time: 2014-01-15 20:42:30
Opening video filter: [vapoursynth file=/script.vpy]
AO: [pulse] 44100Hz stereo 2ch float
[vapoursynth] Could not get script output node.
Video filter chain:
[vd] 1280x720 yuv420p Autoselect/Autoselect CL=1
[in] 1280x720 yuv420p BT.709 (HD)/TV CL=1
[vapoursynth] 1280x720 yuv420p BT.709 (HD)/TV CL=1 <---
[out] ???
Cannot initialize video filters.
Could not initialize video chain.
Video: no video

fuck.mp4 is that "What The F@#% AMV" by the way, don't judge me.

I have an r9 270x if that makes any difference.

@Sperg
Copy link

Sperg commented Nov 2, 2014

Ok I fixed it never mind.

@rr-
Copy link
Member

rr- commented Aug 3, 2015

Using OP's script, for some videos I get

[vapoursynth] Frame requested during init! This is unsupported.
[vapoursynth] Returning black dummy frame with 0 duration.
[vapoursynth] Frame requested during init! This is unsupported.
[vapoursynth] Returning black dummy frame with 0 duration.
zsh: floating point exception (core dumped)  mpv 

and with this slightly changed script:

import vapoursynth as vs

core = vs.get_core()
clip = video_in

if not (clip.width > 1920 or clip.height > 1080 or container_fps > 59):
    clip = core.std.AssumeFPS(clip, fpsnum = int(container_fps * 1e8), fpsden = int(1e8))
    sup = core.mv.Super(clip, pel=2, hpad=0, vpad=0)
    bvec = core.mv.Analyse(sup, blksize=16, isb=True, chroma=True, search=3, searchparam=1)
    fvec = core.mv.Analyse(sup, blksize=16, isb=False, chroma=True, search=3, searchparam=1)
    clip = core.mv.BlockFPS(clip, sup, bvec, fvec, num=60000, den=1001, mode=3, thscd2=12)

clip.set_output()

I get a bit different error message:

[vapoursynth] Frame requested during init! This is unsupported.
[vapoursynth] Returning black dummy frame with 0 duration.
[vapoursynth] Frame requested during init! This is unsupported.
[vapoursynth] Returning black dummy frame with 0 duration.
Attempted to read from a map with error set: Filter creation aborted, zero (unknown) and negative length clips not allowed
zsh: abort (core dumped)  mpv 

@ghost
Copy link

ghost commented Aug 3, 2015

Useless without backtrace...

@rr-
Copy link
Member

rr- commented Aug 3, 2015

Using OP's script:

Program received signal SIGFPE, Arithmetic exception.
0x00007ffff666e822 in ?? () from /usr/lib/libvapoursynth.so
(gdb) bt
#0  0x00007ffff666e822 in ?? () from /usr/lib/libvapoursynth.so
#1  0x00007ffff668fbdd in VSPlugin::invoke(std::string const&, VSMap const&) () from /usr/lib/libvapoursynth.so
#2  0x00007ffff667e680 in ?? () from /usr/lib/libvapoursynth.so
#3  0x00007fffc50f8220 in ?? () from /usr/lib/python3.4/site-packages/vapoursynth.so
#4  0x00007fffe9e2b528 in PyObject_Call (func=func@entry=0x7fffc47bd5a0, arg=arg@entry=0x7fffc47bec50, kw=kw@entry=0x0) at Objects/abstract.c:2040
#5  0x00007fffe9edd4ea in do_call (nk=<optimized out>, na=<optimized out>, pp_stack=0x7fffffffd7d0, func=<optimized out>) at Python/ceval.c:4466
#6  call_function (oparg=<optimized out>, pp_stack=0x7fffffffd7d0) at Python/ceval.c:4264
#7  PyEval_EvalFrameEx (f=0x7fffc5320240, throwflag=<optimized out>) at Python/ceval.c:2838
#8  0x00007fffe9ee0947 in PyEval_EvalCodeEx (_co=0x0, globals=0x0, locals=0x0, args=0x0, argcount=2, argcount@entry=0, kws=0x7fffffffce00, kws@entry=0x0, kwcount=-993783760, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0)
    at Python/ceval.c:3588
#9  0x00007fffe9ee09eb in PyEval_EvalCode (co=<optimized out>, globals=<optimized out>, locals=<optimized out>) at Python/ceval.c:775
#10 0x00007fffc50eedb7 in vpy_evaluateScript () from /usr/lib/python3.4/site-packages/vapoursynth.so
#11 0x00007fffc50efcb5 in vpy_evaluateFile () from /usr/lib/python3.4/site-packages/vapoursynth.so
#12 0x00000000004b4fd9 in drv_vss_load (vf=0x875960, vars=<optimized out>) at ../video/filter/vf_vapoursynth.c:793
#13 0x00000000004b5720 in reinit_vs (vf=vf@entry=0x875960) at ../video/filter/vf_vapoursynth.c:622
#14 0x00000000004b61e3 in config (vf=0x875960, width=1280, height=720, d_width=<optimized out>, d_height=<optimized out>, flags=0, fmt=<optimized out>) at ../video/filter/vf_vapoursynth.c:665
#15 0x00000000004afb63 in vf_reconfig_wrapper (p=<optimized out>, vf=<optimized out>) at ../video/filter/vf.c:591
#16 vf_reconfig (c=0x874d30, params=params@entry=0x7fffffffdf20, override_params=override_params@entry=0x7fffffffde80) at ../video/filter/vf.c:631
#17 0x00000000004ac596 in video_reconfig_filters (d_video=d_video@entry=0x8749b0, params=params@entry=0x7fffffffdf20) at ../video/decode/dec_video.c:411
#18 0x000000000047f997 in filter_reconfig (mpctx=mpctx@entry=0x7f4360, probe_only=probe_only@entry=false) at ../player/video.c:108
#19 0x000000000047fc6b in video_filter (mpctx=mpctx@entry=0x7f4360, eof=eof@entry=false) at ../player/video.c:454
#20 0x000000000047fd02 in video_decode_and_filter (mpctx=mpctx@entry=0x7f4360) at ../player/video.c:495
#21 0x0000000000480ed7 in video_output_image (endpts=-9.2233720368547758e+18, mpctx=0x7f4360) at ../player/video.c:662
#22 write_video (mpctx=mpctx@entry=0x7f4360, endpts=endpts@entry=-9.2233720368547758e+18) at ../player/video.c:800
#23 0x000000000047ca3e in run_playloop (mpctx=mpctx@entry=0x7f4360) at ../player/playloop.c:1028
#24 0x0000000000474d32 in play_current_file (mpctx=0x7f4360) at ../player/loadfile.c:1240
#25 mp_play_files (mpctx=mpctx@entry=0x7f4360) Program received signal SIGABRT, Aborted.

Using my script:

Program received signal SIGABRT, Aborted.
0x00007fffef89e528 in raise () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007fffef89e528 in raise () from /usr/lib/libc.so.6
#1  0x00007fffef89f93a in abort () from /usr/lib/libc.so.6
#2  0x00007ffff6693d99 in vsLog(char const*, long, VSMessageType, char const*, ...) () from /usr/lib/libvapoursynth.so
#3  0x00007ffff667f1d6 in ?? () from /usr/lib/libvapoursynth.so
#4  0x00007fffad2cb039 in ?? () from /usr/lib/vapoursynth/libmvtools.so
#5  0x00007ffff668fbdd in VSPlugin::invoke(std::string const&, VSMap const&) () from /usr/lib/libvapoursynth.so
#6  0x00007ffff667e680 in ?? () from /usr/lib/libvapoursynth.so
#7  0x00007fffc50f8220 in ?? () from /usr/lib/python3.4/site-packages/vapoursynth.so
#8  0x00007fffe9e2b528 in PyObject_Call (func=func@entry=0x7fffc47bd5a0, arg=arg@entry=0x7fffc47bf138, kw=kw@entry=0x7fffc531a448) at Objects/abstract.c:2040
#9  0x00007fffe9edd4ea in do_call (nk=<optimized out>, na=<optimized out>, pp_stack=0x7fffffffd7d0, func=<optimized out>) at Python/ceval.c:4466
#10 call_function (oparg=<optimized out>, pp_stack=0x7fffffffd7d0) at Python/ceval.c:4264
#11 PyEval_EvalFrameEx (f=0x7fffc5399848, throwflag=<optimized out>) at Python/ceval.c:2838
#12 0x00007fffe9ee0947 in PyEval_EvalCodeEx (_co=0x4181, globals=0x4181, locals=0x6, args=0x0, argcount=-135411584, argcount@entry=0, kws=0x4d, kws@entry=0x0, kwcount=-993783760, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0)
    at Python/ceval.c:3588
#13 0x00007fffe9ee09eb in PyEval_EvalCode (co=<optimized out>, globals=<optimized out>, locals=<optimized out>) at Python/ceval.c:775
#14 0x00007fffc50eedb7 in vpy_evaluateScript () from /usr/lib/python3.4/site-packages/vapoursynth.so
#15 0x00007fffc50efcb5 in vpy_evaluateFile () from /usr/lib/python3.4/site-packages/vapoursynth.so
#16 0x00000000004b4fd9 in drv_vss_load (vf=0x875960, vars=<optimized out>) at ../video/filter/vf_vapoursynth.c:793
#17 0x00000000004b5720 in reinit_vs (vf=vf@entry=0x875960) at ../video/filter/vf_vapoursynth.c:622
#18 0x00000000004b61e3 in config (vf=0x875960, width=1280, height=720, d_width=<optimized out>, d_height=<optimized out>, flags=0, fmt=<optimized out>) at ../video/filter/vf_vapoursynth.c:665
#19 0x00000000004afb63 in vf_reconfig_wrapper (p=<optimized out>, vf=<optimized out>) at ../video/filter/vf.c:591
#20 vf_reconfig (c=0x874d30, params=params@entry=0x7fffffffdf20, override_params=override_params@entry=0x7fffffffde80) at ../video/filter/vf.c:631
#21 0x00000000004ac596 in video_reconfig_filters (d_video=d_video@entry=0x8749b0, params=params@entry=0x7fffffffdf20) at ../video/decode/dec_video.c:411
#22 0x000000000047f997 in filter_reconfig (mpctx=mpctx@entry=0x7f4360, probe_only=probe_only@entry=false) at ../player/video.c:108
#23 0x000000000047fc6b in video_filter (mpctx=mpctx@entry=0x7f4360, eof=eof@entry=false) at ../player/video.c:454
#24 0x000000000047fd02 in video_decode_and_filter (mpctx=mpctx@entry=0x7f4360) at ../player/video.c:495
#25 0x0000000000480ed7 in video_output_image (endpts=-9.2233720368547758e+18, mpctx=0x7f4360) at ../player/video.c:662
#26 write_video (mpctx=mpctx@entry=0x7f4360, endpts=endpts@entry=-9.2233720368547758e+18) at ../player/video.c:800
#27 0x000000000047ca3e in run_playloop (mpctx=mpctx@entry=0x7f4360) at ../player/playloop.c:1028
#28 0x0000000000474d32 in play_current_file (mpctx=0x7f4360) at ../player/loadfile.c:1240
#29 mp_play_files (mpctx=mpctx@entry=0x7f4360) at ../player/loadfile.c:1392
#30 0x0000000000475cfc in mpv_main (argc=<optimized out>, argv=0x7fffffffe358) at ../player/main.c:533
#31 0x00007fffef88b790 in __libc_start_main () from /usr/lib/libc.so.6
#32 0x00000000004152e9 in _start ()

Let me know if I should compile vapoursynth with debug symbols.

@ghost
Copy link

ghost commented Aug 3, 2015

Maybe it's some sort of overflow in the clip length calculation. But I think it must be internal to vapoursynth then.

@ghost
Copy link

ghost commented Aug 3, 2015

You can try reducing this number: int enough_for_everyone = INT_MAX / 16; in vf_vapoursynth.c.

@ghost
Copy link

ghost commented Aug 3, 2015

Also don't hijack issues with different issues.

@kabalage
Copy link

kabalage commented Aug 6, 2015

num=60, den=1 instead of 60000, 1001 works until this is fixed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants