WIP: audio mix-minus #126
WIP: audio mix-minus #126
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| /// Implement lightweight audio mixer with mix-minus feature | ||
| /// We will select n highest audio-level tracks | ||
| pub struct AudioMixer<Pkt: Clone, Src: Debug + Clone + Eq + Hash> { | ||
| extractor: Box<dyn (Fn(&Pkt) -> Option<i8>) + Send + Sync>, |
Check warning
Code scanning / clippy
very complex type used. Consider factoring parts into type definitions Warning
| } | ||
|
|
||
| impl<Pkt: Clone, Src: Debug + Clone + Eq + Hash> AudioMixer<Pkt, Src> { | ||
| pub fn new(extractor: Box<dyn (Fn(&Pkt) -> Option<i8>) + Send + Sync>, config: AudioMixerConfig) -> Self { |
Check warning
Code scanning / clippy
very complex type used. Consider factoring parts into type definitions Warning
| pub fn new( | ||
| transport: T, | ||
| mut cluster: C, | ||
| room: &str, | ||
| peer: &str, | ||
| sub_scope: EndpointSubscribeScope, | ||
| bitrate_type: BitrateLimiterType, | ||
| mix_minus_mode: MixMinusAudioMode, | ||
| mix_minus_size: usize, | ||
| ) -> Self { |
Check warning
Code scanning / clippy
this function has too many arguments (8/7) Warning
| if matches!(self.mode, MixMinusAudioMode::AllAudioStreams) && meta.kind.is_audio() { | ||
| if self.mixer.add_source(now_ms, ClusterTrackUuid::from_info(&self.room, peer, track)).is_some() { | ||
| self.outputs.push_back(MediaEndpointMiddlewareOutput::Cluster(ClusterEndpointOutgoingEvent::LocalTrackEvent( | ||
| self.virtual_track_id, | ||
| ClusterLocalTrackOutgoingEvent::Subscribe(peer.clone(), track.clone()), | ||
| ))); | ||
| } | ||
| } |
Check warning
Code scanning / clippy
this if statement can be collapsed Warning
| if matches!(self.mode, MixMinusAudioMode::AllAudioStreams) { | ||
| if self.mixer.remove_source(now_ms, ClusterTrackUuid::from_info(&self.room, peer, track)).is_some() { | ||
| self.outputs.push_back(MediaEndpointMiddlewareOutput::Cluster(ClusterEndpointOutgoingEvent::LocalTrackEvent( | ||
| self.virtual_track_id, | ||
| ClusterLocalTrackOutgoingEvent::Unsubscribe(peer.clone(), track.clone()), | ||
| ))); | ||
| } | ||
| } |
Check warning
Code scanning / clippy
this if statement can be collapsed Warning
| max: value, | ||
| min: value, | ||
| if let Some(consumers) = self.track_sub_map.get(&track_id) { | ||
| for (_, consumer) in consumers { |
Check warning
Code scanning / clippy
you seem to want to iterate on a map's values Warning
| max: bitrate as i64, | ||
| min: bitrate as i64, | ||
| if let Some(consumers) = self.track_sub_map.get(&track_id) { | ||
| for (_, consumer) in consumers { |
Check warning
Code scanning / clippy
you seem to want to iterate on a map's values Warning
| self.consumer_map.insert(consumer.uuid(), track_id); | ||
| self.track_sub_map.insert(track_id, consumer); | ||
| let entry = self.track_sub_map.entry(track_id).or_insert_with(Default::default); |
Check warning
Code scanning / clippy
use of or_insert_with to construct default value Warning
This PR implements an audio mixer with a mix-minus mechanism. This allows for semi-mixed audio in large conference rooms without decoding and encoding audio codecs.
The main idea is to select the tracks with the highest audio levels and send them to the client. These tracks are automatically switched on the server side without any control from the client. Additionally, the client can adjust which source track they are interested in, which is very useful in open-workspace applications like Gather.town.