ffmpeg fails with unexpected number of bytes on gstreamer 1.0 input

[ochilan]

Environment

• v4l2loopback version: 0.12.0
• kernel version: 4.18.12
• Distribution: Arch Linux
• ffmpeg version 4.0.2
• gstreamer version 1.14.4

Problem:

Using gstreamer 1.0 to push video into a v4l2loopback device and consuming it with ffmpeg leads to unexpected number of bytes in buffers in ffmpeg with certain (common?) resolutions/settings.

Repro:

Assuming that /dev/video1 is a v4l2loopback device,

1. gst-launch-1.0 videotestsrc ! video/x-raw,format=YV12,width=1920,height=1080 ! v4l2sink device=/dev/video1
2. ffmpeg -i /dev/video1 -pix_fmt yuv420p -f sdl -

Note that certain other settings such as when using UYVY color + 1920x1200 resolution works.

Observed:

ffmpeg says:
[video4linux2,v4l2 @ 0x5581164f6280] Dequeued v4l2 buffer contains 3112960 bytes, but 3110400 were expected. Flags: 0x00000001.
/dev/video1: Invalid data found when processing input

Expected Results:

The input should be shown.

[umlaeute]

this is a weird one, and i've encountered it before.
my "fix" was, to make the consumer (in your case ffmpeg) accept too large buffers (and just read the expected bytes, ignoring the extraneous ones).

i'm not saying that this is the correct solution, just adding a data point.

[ochilan]

Hmm, interesting. But at least I'm not alone. :) At least some other programs such as OBS seem to work fine with the very same input. Do you know, by any chance, if there is a way to make ffmpeg accept the data without having to hack the code?

[umlaeute]

unfortunately, i do not know.

[ochilan]

I'm digging through the code and found something interesting, maybe you can help me with this one. At two locations, PAGE_ALIGN is used on the device buffer size. In fact, the (aligned) image size is stored as the device's buffer size.

Let us take an example with 1920x1080 pixels and UYVY space (not exactly what is shown in my example above, but same thing, different color space). An image with 1920x1080 pixels in UYVY space should be 2 x 1920 x 1080 = 4147200 bytes in size, but ffmpeg gets 4149248 bytes which is 2048 too much. Indeed, if we align 4147200 bytes to 4096 bytes then we get 4149248 which is the buffer size given to ffmpeg.

Now my question is: Is it correct that PAGE_ALIGN is used on the buffer size and stored as a property of dev? Indeed, this is not the actual expected size of a frame, that should be 4147200 in this example.

EDIT: Hmm, okay, "bytesused" is actually set to the image size in init_buffers(). I wonder if it's ffmpeg's fault not to honor this value then?

[ochilan]

Alright, one more update. First, ffmpeg seems to consider the bytesused value correctly. Also, v4l2loopback sets it correctly in init_buffers().

However, the data from gstreamer is coming in through vidioc_qbuf() and there, bytesused of our buffer is simply set to the value of the incoming buffer. In principle, this makes sense, but the value of bytesused of the buffer coming from gstreamer seems to be "wrong", or at least it is the size of the whole (aligned?) buffer.

My workaround is to set
b->buffer.bytesused = dev->pix_format.sizeimage;
instead of
b->buffer.bytesused = buf->bytesused;
in vidioc_qbuf().