allow listen-only mode in AudioUnit, adjust when category changes #2

davidzhao · 2021-09-10T21:05:48Z

No description provided.

cloudwebrtc · 2021-09-11T10:09:02Z

LGTM

Currently the implementation of FrameTransformers uses distinct, incompatible types for recevied vs about-to-be-sent frames. This adds a flag in the interface so we can at least check that we are being given the correct type. crbug.com/1250638 tracks removing the need for this. Chrome will be updated after this to check the direction flag and provide a javascript error if the wrong type of frame is written into the encoded insertable streams writable stream, rather than crashing. (cherry picked from commit 8fb41a3) Bug: chromium:1247260 Change-Id: I9cbb66962ea0718ed47c5e5dba19a8ff9635b0b1 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/232301 Reviewed-by: Harald Alvestrand <hta@webrtc.org> Commit-Queue: Tony Herre <toprice@chromium.org> Cr-Original-Commit-Position: refs/heads/main@{#35100} Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/233941 Commit-Queue: Harald Alvestrand <hta@webrtc.org> Cr-Commit-Position: refs/branch-heads/4638@{#2} Cr-Branched-From: fb50179-refs/heads/main@{#34960}

…transport at add-track" This reverts commit 7a2db8a. After this commit, the PeerConnection would assume that any new m= sections will be added to the first existing BUNDLE group. This is true of JSEP endpoints (if they don't do SDP munging), but is not necessarily true for non-JSEP endpoints. This breaks the following scenarios: * Remote offer adding a new m= section that's not part of any BUNDLE group. * Remote offer adding a m= section to the second BUNDLE group. The latter is specifically problematic for any application that wants to bundle all audio streams in one group and all video streams in another group when using Unified Plan SDP, to replicate the behavior of using now-deprecated Plan B without bundling. TBR=hta@webrtc.org Bug: webrtc:12837, webrtc:12906, chromium:1236202 Change-Id: I97a348c96443dee95e2b42792b73ab7b101fd64c Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/227681 Reviewed-by: Taylor Brandstetter <deadbeef@webrtc.org> Reviewed-by: Harald Alvestrand <hta@webrtc.org> Commit-Queue: Taylor Brandstetter <deadbeef@webrtc.org> Cr-Commit-Position: refs/branch-heads/4577@{#2} Cr-Branched-From: 5196931-refs/heads/master@{#34463}

…manently. After a send stream is stopped, it can still be re-used and implicitly restarted by activating layers. This change removes marking the flag we use for async operations as 'not alive' inside Stop() and only doing so when the send stream is stopped permanently. The effect this has is that an implicit start via UpdateActiveSimulcastLayers() will run and correctly update the states. Before, if a stream had been stopped, the safety flag would prevent the async operation from running. (cherry picked from commit 51238e6) Bug: chromium:1241213 Change-Id: Iebdfabba3e1955aafa364760eebd4f66281bcc60 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/229304 Commit-Queue: Tommi <tommi@webrtc.org> Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org> Cr-Original-Commit-Position: refs/heads/master@{#34809} Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/229584 Cr-Commit-Position: refs/branch-heads/4606@{#2} Cr-Branched-From: 8b18304-refs/heads/master@{#34737}

the maximum used in practice is multiopus with 6 or 8 channels. 24 is the maximum number of channels supported in the audio decoder. BUG=chromium:1265806 (cherry picked from commit d58ac5a) No-Try: True Change-Id: Iba8e3185a1f235b846fed9c154e66fb3983664ed Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/238980 Reviewed-by: Harald Alvestrand <hta@webrtc.org> Commit-Queue: Philipp Hancke <phancke@nvidia.com> Cr-Original-Commit-Position: refs/heads/main@{#35440} Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/240180 Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org> Commit-Queue: Harald Alvestrand <hta@webrtc.org> Cr-Commit-Position: refs/branch-heads/4692@{webrtc-sdk#2} Cr-Branched-From: c276aee-refs/heads/main@{#35313}

…sion allow listen-only mode in AudioUnit, adjust when category changes

allow listen-only mode in AudioUnit, adjust when category changes

When a display uses a scale factor (different than 1.0) the previous cursor position is not properly cleared during a CRD connection on ChromeOS (see b/235191365). The issue was that the fix for crbug.com/1323241 does not take device scaling into account, so that fix would incorrectly not mark the previous location of the mouse cursor as modified. Adding proper boundary checks is hard and risky though, as the way the position of the mouse cursor is reported seems to be platform dependent (ChromeOS vs Linux vs ...). So because crbug.com/1323241 only solves a theoretical crash that is rarely if ever hit in the field, I decided to for now undo the fix for crbug.com/1323241. A proper boundary check can then later be introduced without any pressure from a looming release (cherry picked from commit ff45105) No-Try: True Bug: chromium:1323241 Bug: b/235191365 Fixed: b/235191365 Test: Manually deployed Change-Id: Ib09b6cc5e396bd52538332edfc4395ed80c6786e Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/265391 Reviewed-by: Alexander Cooper <alcooper@chromium.org> Reviewed-by: Joe Downing <joedow@google.com> Commit-Queue: Jeroen Dhollander <jeroendh@google.com> Cr-Original-Commit-Position: refs/heads/main@{#37274} Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/266491 Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org> Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org> Cr-Commit-Position: refs/branch-heads/5112@{#2} Cr-Branched-From: a976a87-refs/heads/main@{#37168}

allow listen-only mode in AudioUnit, adjust when category changes

This interface will be implemented by "new" ICE controllers that actively manage the connection used by the transport. Bug: webrtc:14367, webrtc:14131 Change-Id: I0858884b0decd2a17ae9ca8617a043a085c61d54 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/271291 Commit-Queue: Sameer Vijaykar <samvi@google.com> Reviewed-by: Jonas Oreland <jonaso@webrtc.org> Cr-Commit-Position: refs/heads/main@{#38066}

allow listen-only mode in AudioUnit, adjust when category changes

This is a simple way to avoid crashing, but the underlying issue of why the entry has been removed, is a bit more complex to fix and will be fixed in a follow-up CL. Using no-try to land since the win_asan bot is failing for an unrelated reason and all other bots have passed. (cherry picked from commit 9c60649) No-try: true Bug: chromium:1374310 Change-Id: I9dc0cf9e1acdcc3b3a205104346cc835b3f79c1b Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/279283 Reviewed-by: Harald Alvestrand <hta@webrtc.org> Commit-Queue: Tomas Gunnarsson <tommi@webrtc.org> Cr-Original-Commit-Position: refs/heads/main@{#38405} Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/280221 Reviewed-by: Artem Titov <titovartem@webrtc.org> Cr-Commit-Position: refs/branch-heads/5359@{webrtc-sdk#2} Cr-Branched-From: fb3bd4a-refs/heads/main@{#38387}

Certificates being missing is a sign of a bug (e.g. webrtc:14844, to be fixed separately) which is why we have a DCHECK. But this DCHECK does not protect against accessing the invalid iterator if it is a release build. This CL makes that safe. # Mobile bots not running properly NOTRY=True (cherry picked from commit 124d7c3) Bug: chromium:1408392 Change-Id: I97a82786028e41c58ef8ef15002c3f959bbec7f1 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/291109 Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org> Commit-Queue: Henrik Boström <hbos@webrtc.org> Cr-Original-Commit-Position: refs/heads/main@{#39159} Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/291380 Cr-Commit-Position: refs/branch-heads/5481@{#2} Cr-Branched-From: 2e1a9a4-refs/heads/main@{#38901}

This prepares TaskQueueBase sub classes to be able to migrate to the location and traits-based API. It re-introduces a Location class into the webrtc namespace, which is meant to be overridden by Chromium. Bug: chromium:1416199 Change-Id: I712c7806a71b3b99b2a2bf95e555b357c21c15ae Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/294381 Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org> Commit-Queue: Markus Handell <handellm@webrtc.org> Reviewed-by: Danil Chapovalov <danilchap@webrtc.org> Reviewed-by: Harald Alvestrand <hta@webrtc.org> Cr-Commit-Position: refs/heads/main@{#39400}

… one which avoids an infinitely growing SDP if the remote end rejects the datachannel section. This will reactivate the m-line even if all datachannels are closed. BUG=chromium:1442604 (cherry picked from commit 522380f) No-Try: True Change-Id: If60f93b406271163df692d96102baab701923602 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/304241 Reviewed-by: Harald Alvestrand <hta@webrtc.org> Commit-Queue: Philipp Hancke <phancke@microsoft.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Original-Commit-Position: refs/heads/main@{#40029} Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/305420 Commit-Queue: Harald Alvestrand <hta@webrtc.org> Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org> Cr-Commit-Position: refs/branch-heads/5735@{#2} Cr-Branched-From: df7df19-refs/heads/main@{#39949}

allow listen-only mode in AudioUnit, adjust when category changes

allow listen-only mode in AudioUnit, adjust when category changes release mic when category changes (#5) Change defaults to iOS defaults (#7) Sync audio session config (#8) * use `AVAudioSession` defaults * remove isRecordingEnabled feat: support bypass voice processing for iOS. (#15) Remove MacBookPro audio pan right code (#22) fix: Fix can't open mic alone when built-in AEC is enabled. (#29) feat: add audio device changes detect for windows. (#41) * feat: add audio device changes detect for windows. * Update audio_device_core_win.cc fix iOS/macOS/Android compile. fix Linux compile (#47) AudioUnit: Don't rely on category switch for mic indicator to turn off (#52) * progress * tweak * clean * simplify audio unit restart call to SetupAudioBuffersForActiveAudioSession() might not be needed since sample rate won't change during restart. This might help reduce the unwanted noise when restarting audio unit. * clean Stop recording on mute (turn off mic indicator) (#55) * initial impl * more comments * more comment * adjust indent * comments Cherry pick audio selection from m97 release (#35) * [Mac] Allow audio device selection (#21) * first attempt * remove unused dep * init playout / recording * use AudioDeviceID as guid * switch device method * equality * default device * `isDefault` property * dont format default device name * type param * bypass * refactor * fix * append Audio to thread labels * ref * lk headers * low level apis * fix thread checks Some methods of ADM needs to be run on worker thread, otherwise RTC's thread check will fail. * switch to default device when removed * close mixerManager if didn't switch to default device * default audio device switched * expose devices update handler * fix ios compile * fix bug: don't always recreate RTCAudioDeviceModule * handle guid. Co-authored-by: Hiroshi Horie <548776+hiroshihorie@users.noreply.github.com>

allow listen-only mode in AudioUnit, adjust when category changes (#2) release mic when category changes (#5) Change defaults to iOS defaults (#7) Sync audio session config (#8) feat: support bypass voice processing for iOS. (#15) Remove MacBookPro audio pan right code (#22) fix: Fix can't open mic alone when built-in AEC is enabled. (#29) feat: add audio device changes detect for windows. (#41) fix Linux compile (#47) AudioUnit: Don't rely on category switch for mic indicator to turn off (#52) Stop recording on mute (turn off mic indicator) (#55) Cherry pick audio selection from m97 release (#35) [Mac] Allow audio device selection (#21) Co-authored-by: Hiroshi Horie <548776+hiroshihorie@users.noreply.github.com> Co-authored-by: David Zhao <dz@livekit.io>

allow listen-only mode in AudioUnit, adjust when category changes (#2) release mic when category changes (#5) Change defaults to iOS defaults (#7) Sync audio session config (#8) feat: support bypass voice processing for iOS. (#15) Remove MacBookPro audio pan right code (#22) fix: Fix can't open mic alone when built-in AEC is enabled. (#29) feat: add audio device changes detect for windows. (#41) fix Linux compile (#47) AudioUnit: Don't rely on category switch for mic indicator to turn off (#52) Stop recording on mute (turn off mic indicator) (#55) Cherry pick audio selection from m97 release (#35) [Mac] Allow audio device selection (#21) RTCAudioDeviceModule.outputDevice / inputDevice getter and setter (#80) Co-authored-by: Hiroshi Horie <548776+hiroshihorie@users.noreply.github.com> Co-authored-by: David Zhao <dz@livekit.io>

allow listen-only mode in AudioUnit, adjust when category changes (#2) release mic when category changes (#5) Change defaults to iOS defaults (#7) Sync audio session config (#8) feat: support bypass voice processing for iOS. (#15) Remove MacBookPro audio pan right code (#22) fix: Fix can't open mic alone when built-in AEC is enabled. (#29) feat: add audio device changes detect for windows. (#41) fix Linux compile (#47) AudioUnit: Don't rely on category switch for mic indicator to turn off (#52) Stop recording on mute (turn off mic indicator) (#55) Cherry pick audio selection from m97 release (#35) [Mac] Allow audio device selection (#21) RTCAudioDeviceModule.outputDevice / inputDevice getter and setter (#80) Allow custom audio processing by exposing AudioProcessingModule (#85) Expose audio sample buffers for Android (#89) feat: add external audio processor for android. (#103) android: make audio output attributes modifiable (#118) Co-authored-by: Hiroshi Horie <548776+hiroshihorie@users.noreply.github.com> Co-authored-by: David Zhao <dz@livekit.io> Co-authored-by: davidliu <davidliu@deviange.net>

allow listen-only mode in AudioUnit, adjust when category changes (#2) release mic when category changes (#5) Change defaults to iOS defaults (#7) Sync audio session config (#8) feat: support bypass voice processing for iOS. (#15) Remove MacBookPro audio pan right code (#22) fix: Fix can't open mic alone when built-in AEC is enabled. (#29) feat: add audio device changes detect for windows. (#41) fix Linux compile (#47) AudioUnit: Don't rely on category switch for mic indicator to turn off (#52) Stop recording on mute (turn off mic indicator) (#55) Cherry pick audio selection from m97 release (#35) [Mac] Allow audio device selection (#21) RTCAudioDeviceModule.outputDevice / inputDevice getter and setter (#80) Allow custom audio processing by exposing AudioProcessingModule (#85) Expose audio sample buffers for Android (#89) feat: add external audio processor for android. (#103) android: make audio output attributes modifiable (#118) Fix external audio processor sample rate calculation (#108) Expose remote audio sample buffers on RTCAudioTrack (#84) Fix memory leak when creating audio CMSampleBuffer #86 Co-authored-by: Hiroshi Horie <548776+hiroshihorie@users.noreply.github.com> Co-authored-by: David Zhao <dz@livekit.io> Co-authored-by: davidliu <davidliu@deviange.net>

Use M125 as the latest version and migrate historical patches to m125 Patches Group: ## 1. Update README.md b6c65fc * Add Apache-2.0 license and some note to README.md. (#9) * Updated readme detailing changes from original (#42) * Adding membrane framework (#51) * Updated readme (#83) ## 2. Audio Device Optimization 7454824 * allow listen-only mode in AudioUnit, adjust when category changes (#2) * release mic when category changes (#5) * Change defaults to iOS defaults (#7) * Sync audio session config (#8) * feat: support bypass voice processing for iOS. (#15) * Remove MacBookPro audio pan right code (#22) * fix: Fix can't open mic alone when built-in AEC is enabled. (#29) * feat: add audio device changes detect for windows. (#41) * fix Linux compile (#47) * AudioUnit: Don't rely on category switch for mic indicator to turn off (#52) * Stop recording on mute (turn off mic indicator) (#55) * Cherry pick audio selection from m97 release (#35) * [Mac] Allow audio device selection (#21) * RTCAudioDeviceModule.outputDevice / inputDevice getter and setter (#80) * Allow custom audio processing by exposing AudioProcessingModule (#85) * Expose audio sample buffers for Android (#89) * feat: add external audio processor for android. (#103) * android: make audio output attributes modifiable (#118) * Fix external audio processor sample rate calculation (#108) * Expose remote audio sample buffers on RTCAudioTrack (#84) * Fix memory leak when creating audio CMSampleBuffer #86 ## 3. Simulcast/SVC support for iOS/Android. b0b9fe9 - Simulcast support for iOS SDK (#4) - Support for simulcast in Android SDK (#3) - include simulcast headers for mac also (#10) - Fix simulcast using hardware encoder on Android (#48) - Add scalabilityMode support for AV1/VP9. (#90) ## 4. Android improvements. 9aaaab5 - Start/Stop receiving stream method for VideoTrack (#25) - Properly remove observer upon deconstruction (#26) - feat: Expose setCodecPreferences/getCapabilities for android. (#61) - fix: add WrappedVideoDecoderFactory.java. (#74) ## 5. Darwin improvements a13ea17 - [Mac/iOS] feat: Add RTCYUVHelper for darwin. (#28) - Cross-platform `RTCMTLVideoView` for both iOS / macOS (#40) - rotationOverride should not be assign (#44) - [ObjC] Expose properties / methods required for AV1 codec support (#60) - Workaround: Render PixelBuffer in RTCMTLVideoView (#58) - Improve iOS/macOS H264 encoder (#70) - fix: fix video encoder not resuming correctly upon foregrounding (#75). - add PrivacyInfo.xcprivacy to darwin frameworks. (#112) - Add NSPrivacyCollectedDataTypes key to xcprivacy file (#114) - Thread-safe `RTCInitFieldTrialDictionary` (#116) - Set RTCCameraVideoCapturer initial zoom factor (#121) - Unlock configuration before starting capture session (#122) ## 6. Desktop Capture for macOS. 841d78f - [Mac] feat: Support screen capture for macOS. (#24) (#36) - fix: Get thumbnails asynchronously. (#37) - fix: Use CVPixelBuffer to build DesktopCapture Frame, fix the crash caused by non-CVPixelBuffer frame in RTCVideoEncoderH264 that cannot be cropped. (#63) - Fix the crash when setting the fps of the virtual camera. (#62) ## 7. Frame Cryptor Support. fc08745 - feat: Frame Cryptor (aes gcm/cbc). (#54) - feat: key ratchet/derive. (#66) - fix: skip invalid key when decryption failed. (#81) - Improve e2ee, add setSharedKey to KeyProvider. (#88) - add failure tolerance for framecryptor. (#91) - fix h264 freeze. (#93) - Fix/send frame cryptor events from signaling thread (#95) - more improvements for E2EE. (#96) - remove too verbose logs (#107) - Add key ring size to keyProviderOptions. (#109) ## 8. Other improvements. eed6c8a - Added yuv_helper (#57) - ABGRToI420, ARGBToI420 & ARGBToRGB24 (#65) - more yuv wrappers (#87) - Fix naming for yuv helper (#113) - Fix missing `RTC_OBJC_TYPE` macros (#100) --------- Co-authored-by: Hiroshi Horie <548776+hiroshihorie@users.noreply.github.com> Co-authored-by: David Zhao <dz@livekit.io> Co-authored-by: davidliu <davidliu@deviange.net> Co-authored-by: Angelika Serwa <angelika.serwa@gmail.com> Co-authored-by: Théo Monnom <theo.monnom@outlook.com>

allow listen-only mode in AudioUnit, adjust when category changes (webrtc-sdk#2) release mic when category changes (webrtc-sdk#5) Change defaults to iOS defaults (webrtc-sdk#7) Sync audio session config (webrtc-sdk#8) feat: support bypass voice processing for iOS. (webrtc-sdk#15) Remove MacBookPro audio pan right code (webrtc-sdk#22) fix: Fix can't open mic alone when built-in AEC is enabled. (webrtc-sdk#29) feat: add audio device changes detect for windows. (webrtc-sdk#41) fix Linux compile (webrtc-sdk#47) AudioUnit: Don't rely on category switch for mic indicator to turn off (webrtc-sdk#52) Stop recording on mute (turn off mic indicator) (webrtc-sdk#55) Cherry pick audio selection from m97 release (webrtc-sdk#35) [Mac] Allow audio device selection (webrtc-sdk#21) RTCAudioDeviceModule.outputDevice / inputDevice getter and setter (webrtc-sdk#80) Co-authored-by: Hiroshi Horie <548776+hiroshihorie@users.noreply.github.com> Co-authored-by: David Zhao <dz@livekit.io>

allow listen-only mode in AudioUnit, adjust when category changes

bf37148

davidzhao requested a review from cloudwebrtc September 10, 2021 21:05

cloudwebrtc approved these changes Sep 11, 2021

View reviewed changes

cloudwebrtc merged commit 1cef977 into main Sep 11, 2021

davidzhao deleted the listen-only-audio-session branch September 11, 2021 18:44

davidliu pushed a commit to davidliu/webrtc that referenced this pull request Feb 8, 2022

Merge pull request webrtc-sdk#2 from webrtc-sdk/listen-only-audio-ses…

1b55c2c

…sion allow listen-only mode in AudioUnit, adjust when category changes

davidliu pushed a commit that referenced this pull request Feb 10, 2022

Merge pull request #2 from webrtc-sdk/listen-only-audio-session

6be2fc2

allow listen-only mode in AudioUnit, adjust when category changes

davidliu pushed a commit that referenced this pull request Jul 17, 2022

Merge pull request #2 from webrtc-sdk/listen-only-audio-session

40e3fc5

allow listen-only mode in AudioUnit, adjust when category changes

davidliu pushed a commit that referenced this pull request Jul 21, 2022

Merge pull request #2 from webrtc-sdk/listen-only-audio-session

fb9de41

allow listen-only mode in AudioUnit, adjust when category changes

vickyleu referenced this pull request Jul 28, 2022

fix issue for screen sharing.

e958e66

cloudwebrtc added a commit that referenced this pull request Jan 18, 2023

Merge pull request #2 from webrtc-sdk/listen-only-audio-session

91945a0

allow listen-only mode in AudioUnit, adjust when category changes

cloudwebrtc added a commit that referenced this pull request Jun 5, 2023

Merge pull request #2 from webrtc-sdk/listen-only-audio-session

46c451d

allow listen-only mode in AudioUnit, adjust when category changes

cloudwebrtc mentioned this pull request Jun 12, 2023

update to m114 #76

Merged

cloudwebrtc mentioned this pull request May 21, 2024

Update to m125. #119

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

allow listen-only mode in AudioUnit, adjust when category changes #2

allow listen-only mode in AudioUnit, adjust when category changes #2

davidzhao commented Sep 10, 2021

cloudwebrtc commented Sep 11, 2021

allow listen-only mode in AudioUnit, adjust when category changes #2

allow listen-only mode in AudioUnit, adjust when category changes #2

Conversation

davidzhao commented Sep 10, 2021

cloudwebrtc commented Sep 11, 2021