Video corruption: Don't drop video keyframes

[ovalseven8]

Originally posted by @tux3 in irungentoo#1479:

Currently as I understand it toxcore shoves all the video data over UDP and hopes it reaches the destination.

This creates horrible video corruption when a key frame is lost. On desktop streaming, intra frames are super rare so we can easily go 10s at a time with completely garbled video output. It's a common thing when calling EchoBot for example, with both uTox and qTox.

Toxcore should try on the sending end to retransmit iframes until it either gets an ACK or the next iframe is ready, and on the receiving end it shouldn't give Tox clients inter frames that depend on missing intra frames.

For more information: qTox/qTox#2529

[alexeysvrv]

@mannol Please help

[GrayHatter]

For anyone watching, I'm not sure keyframes are that important anymore.

And either way, there's a lot more work than just this one issue.

[alexeysvrv]

@GrayHatter Why keyframes are not important? Video and so will be broken?

[isotoxin]

Isotoxin currently uses lossless video transferring (only Isotoxin-Isotoxin calls, obviously), so it usable to share huge resolution desktop with good quality.
Lossy video transfering is horrible. A fortiori, lossy way of toxcore. Do you think toxcore lossy is just sending without confirmation of sending was successful? No way! Toxcore lossy is DON'T SEND PACKET WHEN SENDING QUEUE IS FULL! What a terrible solution...

[GrayHatter]

@alexeysvrv sending JUST keyframes isn't the issue. The problem is toxav doesn't try to recover from a high data loss event, or temporary network instability. It's currently a 'dumb' protocol, that tries it's best, and never adapts.

Fixing the sending of just keyframes will solve a single issue, one that's not often reported. That is to say, if we magically never drop a keyframe, video quality will still be problematic.

[alexeysvrv]

@GrayHatter What is the solution?