MJPEG stream decoding problem

[tjf]

Hello,

I am struggling trying to open a MJPEG stream off an IP camera that opens without issue using ffplay. This happens with any MJPEG stream in my experience thus far.

I am receiving AVError: [Errno 1094995529] Invalid data found when processing input when calling decode() on the Packet instance. I am using the libav from ffmpeg 2.6.3 on MacOSX.

Not 100% sure if this is a bug or just operator error. The URL in my example is a demo URL from an IP camera manufacturer that displays the same problem.

Any thoughts?

For example:

import av
import logging
logging.basicConfig()

container = av.open("http://50.197.211.181:8905/videostream.cgi?user=user&pwd=foscam")
video_stream = next(s for s in container.streams if s.type == b'video')

for packet in container.demux(video_stream):
    for frame in packet.decode():
        print "Captured frame!"

outputs:

WARNING:libav.mjpeg:Format mjpeg detected only with low score of 25, misdetection possible!
WARNING:libav.mjpeg:ignoring invalid SAR: 0/0
Traceback (most recent call last):
  File "/Users/tjf/camera/test.py", line 9, in <module>
    for frame in packet.decode():
  File "av/packet.pyx", line 30, in av.packet.Packet.decode (src/av/packet.c:1310)
  File "av/stream.pyx", line 203, in av.stream.Stream.decode (src/av/stream.c:4048)
  File "av/video/stream.pyx", line 51, in av.video.stream.VideoStream._decode_one (src/av/video/stream.c:1716)
  File "av/utils.pyx", line 32, in av.utils.err_check (src/av/utils.c:1012)
av.AVError: [Errno 1094995529] Invalid data found when processing input

[mikeboers]

I get a 401 Unauthorized on that URL. Another example?

[tjf]

That's odd.

I was just able to run ffplay -i "http://50.197.211.181:8905/videostream.cgi?user=user&pwd=foscam" without a problem.

Be sure you're quoting the URL as the ampersand gets caught by the shell and that would lead to the 401 you're experiencing. Honestly, I tried several MJPEG streams from public webcams all over the net with the same issue.

[mikeboers]

That was a bit silly of me...

Anyways, I can confirm the behaviour.

The error is raised when we try to flush the stream in this loop. A frame has already been decoded when the exception is thrown.

Options to take here:

1. Eat any errors we encounter when decoding frames beyond the first. But what happens when there is a legitimate error?
2. Expose option to only decode one frame. But what happens with a codec that builds up a buffer until it errors? I guess we put a warning on that option and try to expose a buffer size.

I'm leaning towards 2 immediately.

[mikeboers]

With the current master, I can do:

import av
import logging
logging.basicConfig()

container = av.open("http://50.197.211.181:8905/videostream.cgi?user=user&pwd=foscam")
video_stream = next(s for s in container.streams if s.type == b'video')

for packet in container.demux(video_stream):
    while True:
        frame = packet.decode_one()
        if frame is None:
            break
        print frame

[tjf]

Hi, Mike.

Pardon my ignorance, but I would like to be sure I understand the issue correctly.

Is the issue that the packet does not contain enough data to decode the second frame which causes the error to be raised in that loop?

Would using decode_one on other codecs effectively drop additional frames that may be in the same packet?

Thanks for your time and help with this. I really appreciate it!

[mikeboers]

In general, I'm not 100% on the implications of the changes I made, and I see there is an error in the example I provided you. Since we reset the packet buffers after we decode from them, continuing to decode one from your MJPEG results in the same frame coming out infinitely. The example should really be:

for packet in container.demux(video_stream):
    print packet
    print packet.decode_one()

PyAV would extract as many frames as it could from each packet, just in case there were multiple frames in there. Otherwise, a buffer will eventually fill up and error.

In this case, for whatever reason, that flushing procedure results in an exception being raised.

In MJPEG you can safely assume there is only one frame per packet. It will actually be a complete JPEG. In other codecs... not so much.

In general, we should consider if we should mutate a packet after consuming frames from it. I honestly don't know enough to make that decision now. But this commit as is should work for you.

[tjf]

That's good information, thanks.

I'm typically in the 'immutable camp', so I'd not mutate the packet after partially decoding it, if at all possible. I can take a stab at looking at this a little deeper as well.

Should I close the issue, or would you like to keep it open for a more permanent fix or refactor?

[mikeboers]

Lets leave it open until we manage to reformulate our new questions into a new issue.

[mkassner]

I have the same problem, but I find a fix like this in stream.pyx simpler:

pupil-labs@94a2c67

Why not like this?

[mkassner]

I just wanted to add that I have been using my fix successfully for a while now.

I'd make a case for this because the fact that the current implementation fails to read mjpeg and only works when running the decode process via decode_one on the user side seems cumbersome. Decode should just work regardless of the codec used. The condition (codec==mjpeg) is easy to check for during decoding so why bother the lib user about this implementation detail?

If its desired I'll create a PR.

[mikeboers]

I agree with you that if the codec will only ever have a single frame per packet, to not bother trying for more. (I'd check again, but) MJPEG does seem to be that way. I wonder if there is already a flag or something on the codec descriptor...