Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add VP8 simulcast stream bitrate calculator #1427

Closed
wants to merge 1 commit into from

Conversation

dsmeytis
Copy link
Contributor

If WebRTC client's uplink is bad to a degree that encoder periodically sends key frames
of the higher spatial layer but it is not enough for stable playback it leads to a situation
when higher layer keyframe switches the stream however no packets are sent afterwards
and picture freezes until the next keyframe appears instead of keep on sending stable lower layer stream.

Steps to reproduce:

  1. Start video conference in 720p highest spatial layer
  2. Limit uplink for some webrtc client to 512kbit/s
  3. Observe "slideshow"-like video from that peer on other participants' screens.

In order to solve this issue I would like to introduce simple bitrate calculator for "middle" and "high" spatial layers.
So it would upscale only if such layer is received at least at minimal bitrate (thresholds are got from webrtc source code, see simulcast.cc)

If WebRTC client's uplink is bad to a degree that encoder periodically sends key frames
of the higher spatial layer but it is not enough for stable playback it leads to a situation
when higher layer keyframe switches the stream however no packets are sent afterwards
and picture freezes until the next keyframe appears instead of keep on sending stable lower layer stream.
@jitsi-jenkins
Copy link

Hi, thanks for your contribution!
If you haven't already done so, could you please make sure you sign our CLA (https://jitsi.org/icla for individuals and https://jitsi.org/ccla for corporations)? We would unfortunately be unable to merge your patch unless we have that piece :(.

@bgrozev
Copy link
Member

bgrozev commented Sep 10, 2020

Hi @dsmeytis , thank you for the contribution!

We already calculate the bitrates of each layer in the receive pipeline, and this information is available to BitrateController when it makes layer selection decisions. I think this would be a more appropriate place for the logic that you propose (though I'm not sure how to apply it for VP8 simulcast only) -- if a layer has a suspiciously low bitrate, do not select it at all.

Can you tell us more about the issue that you experience? You observe the browser sending only keyframes on the high layer?

@dsmeytis
Copy link
Contributor Author

Hi @bgrozev , thank you for the quick response.
I guess it would be better to show an example. Please, take a look at the screenshot
Screenshot
Red line represents the traffic from JVB to some peer and other lines are simulcast layer streams from another peer.
So between 55 and 80 seconds I limited uplink to 512 kbit/s. And as you can see the lowest layer is transmitted fine and sometimes there are some packets from the middle layer. As far as I understood the root cause, WebRTC client's bandwidth estimator tries increase bitrate and it gets enough to send next layer, however when it starts doing that packet losses gets more than 10% and it falls back to the point where it can't encode the layer. So JVB receives only few packets, especially keyframes and tries to switch the layer for its receivers.

Thank you for your hint about BitrateController, I have somehow missed that point. I agree it is better place for that I will take a look at that piece of code.

@dsmeytis
Copy link
Contributor Author

Better solution proposed in PR#1439

@dsmeytis dsmeytis closed this Sep 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants