Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reducing audio packet rate while track is disabled #1764

Open
fippo opened this issue Feb 5, 2018 · 12 comments

Comments

@fippo
Copy link
Contributor

commented Feb 5, 2018

spawned from webrtc/samples#1009

http://w3c.github.io/webrtc-pc/#dom-rtcrtpsender-track talks about about reducing the frame rate for muted video.

Is there any reason why the same doesn't apply to audio? DTX reduces the packet rate drastically and while the audio is muted (by the user) there is a clear indication that we don't need to encode audio.

@alvestrand

This comment has been minimized.

Copy link
Contributor

commented Feb 6, 2018

The "one black frame per second" language was added very recently; discussion in the group revealed that some people felt uneasy about some implementations just sending a black frame and then no more video data for a long time, while others felt that sending at full frame rate was just silly. This got added both to advise senders on what to do and to advise receivers on what they can expect.

I don't think audio codecs have the idea of "frame rate" (they use "sample rate", which is something quite different), so the same language can't be used. Suggestions for language that is reasonable for all audio codecs?

@fippo

This comment has been minimized.

Copy link
Contributor Author

commented Feb 6, 2018

https://tools.ietf.org/html/rfc6716#section-2.1.9 actually uses frame.
Language is really tricky here since opus-DTX should be recommended by default as it can have a negative impact on voice quality - not if muted though.

I would very much like to avoid having to use replaceTrack just to reduce the packet rate (but I will if I have to)

@alvestrand

This comment has been minimized.

Copy link
Contributor

commented Feb 8, 2018

We'd like to have people use track.enabled = false for mute, and that codecs do the Right Thing to reduce packet rate when the track is silent. @fluffy may have comments here too.

@alvestrand

This comment has been minimized.

Copy link
Contributor

commented Feb 8, 2018

RFC 3389 about comfort noise:

The CN packet update rate is left implementation specific. For
example, the CN packet may be sent periodically or only when there is
a significant change in the background noise characteristics. The
CNG algorithm at the receiver uses the information in the CN payload
to update its noise generation model and then produce an appropriate
amount of comfort noise.

@aboba

This comment has been minimized.

Copy link
Contributor

commented Apr 19, 2018

@fippo Can you respond?

@nils-ohlmeier

This comment has been minimized.

Copy link

commented Apr 19, 2018

I think the major difference between video and audio here are that video (usually) keeps state on the screen. So if you stop sending video out of the blue, the user would see a frozen picture. That why it makes more sense to replace that information with intended black.

But with audio there is not confusing last information rendered until replaced. Thus I would actually suggest to simply send nothing. Because in my opinion comfort noise is something which is not supported by all audio codecs and usually needs to be negotiated on top. Thus suggesting CN or DTX would make this features required to be supported.

@aboba

This comment has been minimized.

Copy link
Contributor

commented Apr 20, 2018

CN is mandatory-to-implement in WebRTC, and we have RTCRtpEncodingParameters.dtx so there are multiple ways to control whether discontinuous transmission is enabled or not. This seems like a more straightforward way to control the bitrate than relying on side-effects of setting sender.track.enabled to false.

@fippo

This comment has been minimized.

Copy link
Contributor Author

commented Apr 20, 2018

DTX has to do some magic to detect silence and this was recently quite broken in Chrome, delaying speech for 1+ second. Giving a clear indication that the track was muted might help the encoder here.

Arguably that is an implementation detail but so is reducing the frame rate for muted video.

@stefhak

This comment has been minimized.

Copy link
Contributor

commented Jul 19, 2018

@fippo can you propose (perhaps in form of a PR) language?

I must admit I'm not convinced we need to do anything more here.

@aboba

This comment has been minimized.

Copy link
Contributor

commented Aug 8, 2019

Currently the text in Section 5.2 says that the "RTCRtpSender MUST send silence". This would seem to imply that the DTX setting or CN negotiation state has no effect.

@dontcallmedom

This comment has been minimized.

Copy link
Member

commented Sep 25, 2019

RESOLUTION: Accept Nils proposal for Issue 1764

Nils proposal:

  • Video keeps state on the screen, so it is better to send a black frame than to send nothing andhave a frozen picture. This is not the case for audio.
  • Recommendation: send nothing regardless of CN or DTX (since they may not be negotiated).
@jan-ivar

This comment has been minimized.

Copy link
Contributor

commented Oct 2, 2019

@aboba to provide PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
7 participants
You can’t perform that action at this time.