[Feature request?] Play multiple videos sides by sides

[ArchangeGabriel]

Hi everyone,

Maybe this is not the right place for this, or maybe this feature already exist and I’ve missed it or they are existing third-party tool allowing this, in which case I suppose someone will be kind enough to show me the right direction. ;)

I have a use case where I would like to play multiple videos side by side (OK, until there nothing my WM can’t do), but synchronously (and that’s the thing). They all have the same duration and resolution (or at least aspect ratio).

Ideally, I would like to choose how the videos are placed with regard to each other, but I could definitively live with a fixed setup for 2, 3 and 4 streams (which I consider a reasonable limit for the feature to be useful, unless you have a very big screen).

In my case, the splitting for 2 streams would be left/right halves of the screen for instance, but maybe it could be upper/lower ones depending on the aspect ratio in other cases. For 4 streams, 4 corners, but again that could depend on aspect ratio. For 3 stream, maybe 4 corners with a black one. Ditto.

Since this looks like lot of work in mpv for a certainly not high priority feature (if desirable at all), I was wondering whether this splitting could be left at the user responsibility (which can then use whatever — tiling for instance — WM he wants), while providing a way to synchronise multiple mpv instances.

Is there a way to do so? Else, could it be done?

Thanks in advance, and keep doing great work!

[kokoko3k]

Maybe it is already possible through --lavfi-complex option.
check:
https://trac.ffmpeg.org/wiki/Create%20a%20mosaic%20out%20of%20several%20input%20videos

[ghost]

Yes, you can make mpv open several files with --external-file or whatever it was, and then stack them together using lavfi-complex.

Probably slow.

[wiiaboo]

There's also https://github.com/Syncplay/syncplay.

[ArchangeGabriel]

Awesome! “Probably slow” is no issue for me, it’s 1000×800@5fps streams that I have. ;) Thanks both of you for your help.

So, if anyone is looking for this, here is what I ended up doing:

• 2 videos: mpv <video1> --external-file=<video2> --lavfi-complex='[vid1] [vid2] hstack [vo]'
• 4 videos: mpv <video1> --external-file=<video2> --external-file=<video3> --external-file=<video4> --lavfi-complex='[vid1] [vid2] hstack [t1] ; [vid3] [vid4] hstack [t2] ; [t1] [t2] vstack [vo]'

hstack can be changed to vstack and reciprocally, depending on what you want to achieve in terms of disposition. ;)

I consider this as solved, thanks again!

P.S.: Side question, while doing this I’ve discovered that hwdec (VAAPI here) wasn’t working for yuv444p streams (it’s more obvious when written 4 times than one), I suppose this is libva or hardware support issue?

[wiiaboo]

Non-yuv420p hwaccel support is rare/nonexistent.

[ArchangeGabriel]

@wiiaboo Thanks.

Closing this as solved. :)

[ArchangeGabriel]

Oh, and one other thing: is there a way to avoid this ugly multiple --external-file=? I’ve tried multiple things like , or : separators between files, to no avail. Should I open a new FR for external-file to support multiple files, or did I miss something again?

[wiiaboo]

--external-file={video1,video2}?

[ghost]

I don't think it does separators.

[ArchangeGabriel]

@wiiaboo That did it, thanks! Wonder why I haven’t tried that…

Funny thing: this doesn’t work with hwdec anyway. After converting my videos to yuv420p (well, generating them directly to this format to be exact), I’ve stumbled upon this:

[ffmpeg] Impossible to convert between the formats supported by the filter 'mpv_src_vid1' and the filter 'auto-inserted scaler 0'
failed to configure the filter graph

And then mpv stops. But as the doc says:

Most video filters will not work with hardware decoding as they are primarily implemented on the CPU. Some exceptions are vdpaupp, vdpaurb and vavpp. See VIDEO FILTERS for more details.

Let’s use vaapi-copy. ;) Thanks again everyone for your great and appreciated help! :)

[nvmnghia]

@ArchangeGabriel is there a way to do this side by side compare of videos of different resolutions?

[qmega]

is there a way to do this side by side compare of videos of different resolutions?

You can use a scale filter to get one to the size of the other, or scale each to half your screen, or whatever. The manual does mention this, but here are a couple examples:

mpv somevideo720.mkv --external-file=somevideo1080.mkv --lavfi-complex='[vid1] scale=1920x1080 [vid1_scale]; [vid1_scale][vid2] hstack [vo]'

mpv somevideo720.mkv --external-file=somevideo1080.mkv --lavfi-complex='[vid1] scale=960x540 [vid1_scale]; [vid2] scale=960x540 [vid2_scale]; [vid1_scale][vid2_scale] hstack [vo]'

That won't use mpv's own high-quality gpu scaling. You can change the scaling algorithm ffmpeg uses by giving flags to the scale filter:

mpv somevideo720.mkv --external-file=somevideo1080.mkv --lavfi-complex='[vid1] scale=1920x1080:flags=spline [vid1_scale]; [vid1_scale][vid2] hstack [vo]'

mpv by default will still re-scale the result if your window size is different from that of the combined videos. You can prevent that with --video-unscaled.

[nvmnghia]

is there a way to do this side by side compare of videos of different resolutions?

You can use a scale filter to get one to the size of the other, or scale each to half your screen, or whatever. The manual does mention this, but here are a couple examples:

mpv somevideo720.mkv --external-file=somevideo1080.mkv --lavfi-complex='[vid1] scale=1920x1080 [vid1_scale]; [vid1_scale][vid2] hstack [vo]'

mpv somevideo720.mkv --external-file=somevideo1080.mkv --lavfi-complex='[vid1] scale=960x540 [vid1_scale]; [vid2] scale=960x540 [vid2_scale]; [vid1_scale][vid2_scale] hstack [vo]'

That won't use mpv's own high-quality gpu scaling. You can change the scaling algorithm ffmpeg uses by giving flags to the scale filter:

mpv somevideo720.mkv --external-file=somevideo1080.mkv --lavfi-complex='[vid1] scale=1920x1080:flags=spline [vid1_scale]; [vid1_scale][vid2] hstack [vo]'

mpv by default will still re-scale the result if your window size is different from that of the combined videos. You can prevent that with --video-unscaled.

I find the filters complex. Is it similar to ffmpeg's filter_complex or lavfi?

[qmega]

Is it similar to ffmpeg's filter_complex or lavfi?

It is exactly the same as that, AFAIK.

Filtergraph syntax is documented in ffmpeg-filters(1).
mpv-specific bits are documented in its own manual.

[ghost]

The label names (for track selection) are different, otherwise the graph syntax is the same.

[ruirigel]

Awesome! “Probably slow” is no issue for me, it’s 1000×800@5fps streams that I have. ;) Thanks both of you for your help.

So, if anyone is looking for this, here is what I ended up doing:

• 2 videos: mpv <video1> --external-file=<video2> --lavfi-complex='[vid1] [vid2] hstack [vo]'
• 4 videos: mpv <video1> --external-file=<video2> --external-file=<video3> --external-file=<video4> --lavfi-complex='[vid1] [vid2] hstack [t1] ; [vid3] [vid4] hstack [t2] ; [t1] [t2] vstack [vo]'

hstack can be changed to vstack and reciprocally, depending on what you want to achieve in terms of disposition. ;)

I consider this as solved, thanks again!

P.S.: Side question, while doing this I’ve discovered that hwdec (VAAPI here) wasn’t working for yuv444p streams (it’s more obvious when written 4 times than one), I suppose this is libva or hardware support issue?

At this moment i have 3 ipcams i would like to view them in the same window but i am having trouble understanding if it is a limitation or the construction of my command.

mpv rtsp://user:pw@address:551/stream2 --external-file=rtsp://user:pw@address:552/stream2 --external-file=rtsp://user:pw@address:553/stream2 --lavfi-complex='[vid1] hstack [t1] ; [vid2] hstack [t2] ; [vid3] hstack [t3] ; [t1] [t2] [t3] vstack [vo]'

[ffmpeg] AVFilterGraph: Too many inputs specified for the "vstack" filter.
parsing the filter graph failed

i can play simultaneously on two cameras. any ideas?

[richardpl]

this is not ffmpeg help channel. vstack default number of inputs is 2 if not specified.
Doing cascaded stacking is much faster with single xstack filter

[stefs22]

Are anyone know how to do 4 streams at once?

[ruirigel]

Are anyone know how to do 4 streams at once?

mpv rtsp://login:pw@address --external-file=rtsp://login:pw@address --external-file=rtsp://login:pw@address --lavfi-complex="[vid1][vid2][vid3][vid4]hstack=inputs=4[vo]"

[tg-m]

4 input videos + scaling + sane positioning

Based on: https://trac.ffmpeg.org/wiki/Create%20a%20mosaic%20out%20of%20several%20input%20videos

f1=file1.mp4
f2=file2.mp4
f3=zz.mp4
f4=last.mp4

xsize=960
ysize=540
scale="${xsize}x${ysize}"
full="$((2*${xsize}))x$((2*${ysize}))"

mpv \
    ${f0} \
    --external-file=${f1} \
    --external-file=${f2} \
    --external-file=${f3} \
    --lavfi-complex="
        nullsrc=size=${full} [base];
        [vid1] setpts=PTS-STARTPTS, scale=${scale} [upleft];
        [vid2] setpts=PTS-STARTPTS, scale=${scale} [upright];
        [vid3] setpts=PTS-STARTPTS, scale=${scale} [downleft];
        [vid4] setpts=PTS-STARTPTS, scale=${scale} [downright];
        [base][upleft] overlay=shortest=1 [tmp1];
        [tmp1][upright] overlay=shortest=1:x=${xsize} [tmp2];
        [tmp2][downleft] overlay=shortest=1:y=${ysize} [tmp3];
        [tmp3][downright] overlay=shortest=1:x=${xsize}:y=${ysize} [vo]
    "

[richardpl]

This will break horribly with timestamp gaps between files, and within files.

[tg-m]

Do you suggest that it would be better to set PTS to something like N/(30*TB) or maybe not setting it at all?

[richardpl]

If you do not set it than I guess you will not like output of video.