feat(audio): add experimental transparent listen only mode

[prlanzarin]

What does this PR do?

• feat(audio): add experimental transparent listen only mode
  • Refurbishing Proof-of-concept: phase out listen only #17765 into an actual PR
  • This is an initial, experimental implementation of the feature proposed in
    Microphone: phase out listen only, make it transparent #14021.
• build(bbb-webrtc-sfu): v2.11.0-beta.1

Closes Issue(s)

None, relates to #14021

Motivation

The intention is to phase out the explicit listen only mode with two
overarching goals:

• Reduce UX friction and increase familiarity: the existence of a separate
  listen only mode is a source of confusion for the majority of users
  Reduce average server-side CPU usage while also making it possible for
  having full audio-only meetings.

The proof-of-concept works based on the assumption that a "many
concurrent active talkers" scenario is both rare and not useful. With
that in mind, this includes two server-side triggers:

• On microphone inactivity (currently mute action that is sustained for
  4 seconds, configurable): FreeSWITCH channels are held (which translates
  to much lower CPU usage, virtually 0%). Receiving channels are switched,
  server side, to a listening mode (SFU, mediasoup).
  • This required an extension to mediasoup two allow re-assigning producers
    to already established consumers. No re-negotiation is done.
• On microphone activity (currently unmute action, immediate):
  FreeSWITCH channels are unheld, listening mode is deactivated and the
  mute state is updated accordingly (in this order).

More

This is off by default. It needs to be enabled in two places:

• /etc/bigbluebutton/bbb-webrtc-sfu/production.yml ->
  transparentListenOnly: true
• End users:
  • Server wide: /etc/bigbluebutton/bbb-html5.yml ->
    public.media.transparentListenOnly: true
  • Per user: userdata-bbb_transparent_listen_only=true

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

[antobinary]

Well done @prlanzarin ! I was just testing this and it's amazing! 👏

[shabeepk]

Hi,

I have upgraded my 2.6 installation to 2.7.0-rc1 tried enabling transparent listen only mode buy it doesn't seem to have any effect on the user experience. Is there something I am missing?

I have set:

/usr/share/meteor/bundle/programs/server/assets/app/config/settings.yml:
transparentListenOnly: true

/etc/bigbluebutton/bbb-webrtc-sfu/production.yml
mediasoup:
webrtc:
listenIps:
- ip: 0.0.0.0
announcedIp: public ip
transparentListenOnly: true
plainRtp:
listenIp:
ip: 0.0.0.0
announcedIp: public ip
transparentListenOnly: true
transparentListenOnly: true

/etc/bigbluebutton/bbb-html5.yml was empty, it now has just one line:
public.media.transparentListenOnly: true

But still so far I am unable to see any difference and it seems to be not working.

What am I missing?
How to enable transparent listen only mode in 2.7.0-rc1

[prlanzarin]

@shabeepk Nothing should change visually to end users with just transparentListenOnly: true. This PR adds the backend groundwork for this experimental mode to be tested - and that's it. The difference you're looking for is in operational metrics (e.g.: FreeSWITCH CPU usage per audio channel).

You can change things visually if you want - for instance: disable listen only and force microphone/audio activation on join (both have settings.yml and userdata flags to control). See https://docs.bigbluebutton.org/administration/customize/#application-parameters.

[shariquealavi]

Ohh... I was mistaken badly it seems. You've pointed me to the right direction. I have turned skipCheck: true, listenOnlyMode: false. and it kind of seems to give that behavior that I was looking for. I have also enableDynamicAudioDeviceSelection: true hoping that it may allow the user to seamlessly switch between mic and listen only but it seems this option isn't doing that aswell.

However, I have somewhat got what I was looking for thought if it was something where a user (not moderator) joins in listen-only automatically and has an option to turn mic on on-demand seamlessly ... that would have been awesome.

[shariquealavi]

@prlanzarin The reason being, if I simply disable listen-only ... it won't let a user join the meeting if they haven't given mic permissions... Ideally it should fallback to listen-only for those users whose mic permission isn't available.

Just as it would fallback to listen-only if I lock all the user's mics away.

Is there a way to do that ?

[prlanzarin]

@prlanzarin The reason being, if I simply disable listen-only ... it won't let a user join the meeting if they haven't given mic permissions... Ideally it should fallback to listen-only for those users whose mic permission isn't available.

Just as it would fallback to listen-only if I lock all the user's mics away.

Is there a way to do that ?

@shariquealavi Not right now, there isn't. It's planned, though - one of the follow up items outlined in #14021 (comment) ( Allow enabling a microphone device for an ongoing audio session). We'll work our way to that eventually - you can follow that issue as we'll try to reference PRs in that one as they come in.

[shariquealavi]

@prlanzarin Alright, that's good news. I look forward to it.

I must say you guys are doing the best work out there, I am a fan. 👍 😄