Streaming from the net introduces latency or requires separate thread

[phoboslab]

When feeding a plm_buffer() from the network (or slow media) there's no way to tell the buffer that no data is available at this time, but may be available later.

This forces you to either only decode a frame whenever you are sure that the data for the whole frame is available (as the video decoder can't pause in the middle of a frame) or to run the decoder in a separate thread and busy wait in the plm_buffer_callback until more data is available.

Making sure that enough data is available for decoding a full frame is not straight forward, because we don't know the size of a frame until it has been fully decoded, or we find the PICTURE_START code of the next frame. This introduces unnecessary latency for streaming.

The problem is described in more detail in this blog post towards the end.

This issue is meant for discussion of the problem and possible solutions.

[andyjpb]

Perhaps a non-blocking-io pattern would be useful here. The demuxer can return either a packet or EAGAIN. If the decoder gets EAGAIN from the demuxer then it does nothing and waits to be called again.
This would probably involve changing the interface for the decoder as well so that the user can ask for a frame if one is ready or they can ask the decoder to block until the next frame is ready.

I guess that the wire format doesn't include the data size because doing so would force a frame of latency at the encoder whereas this way the decoder has more implementation options.