Distorted Sound when receiving recorded audioframe

[winstondu]

In summary: Check out https://github.com/winstondu/Agora-RecordFrame-Bug. This project lightly modifies @CavanSu 's example code to add voice recording functionality, in order to demonstrate the issue I am talking about.

Background
Our app uses the chatRoomGaming Audio Profile in order to filter out background game sounds and other noises from the chat.

Currently, we want a method of recording the local user's voice (and only the local user's voice) when they are on a call. While we could obviously do so by tapping the unprocessed audio from iOS, we want to record the Agora-processed version of the local user's microphone audio that used Agora's chatRoomGaming audio profile filters.

In other words, we imagine the flow to be like this:

Unprocessed Mic Audio ----------(Agora Processing with profile) -------------> Processed Mic Audio (we want this) ---------> Agora Servers

I looked into the AudioFrameObserver api. In particular the onRecordAudioFrame function should have done the trick.

However, there is (what I believe to be) a bug: because I called setAudioProfile(.musicStandardStereo, scenario: .chatRoomGaming), the recorded voice is extremely distorted. It is a bug because if I never called the setAudioProfile command, the recorded audio is completely fine. In fact, if I change the setAudioProfile call even slightly, to setAudioProfile(**.speechStandard**, scenario: .chatRoomGaming), the recorded audio is fine as well.

The thing is, I can't not call setAudioProfile, since not calling it would mean I no longer benefit from the .chatRoomgGaming audio profile's background noise suppression on my Agora calls.

Sample Audio File.
In the following file, I recorded myself saying the words "testing, testing, testing", and this was the result:
IMG_1513.mp4.zip)

Smartphone (please complete the following information):

• Device: any iPhone
• OS: iOS 13.6
• Version: Agora iOS 3.0.1.1

[winstondu]

@CavanSu , is there something I need to change to fix the voice distortion issue (seen in https://github.com/AgoraIO/Advanced-Audio/files/5127757/IMG_1513.mp4.zip)? Your Speech recognizer example only did remote audio, not the local recorded audio

[CavanSu]

Can you write audio data to pcm file directly? you upload mp4 file's sampleRate seems wrong @winstondu

[winstondu]

@CavanSu , do you have a code snippet that illustrates how I can write to PCM file directly?

also, were you able to replicate my issue using my example project?

[winstondu]

@CavanSu , I finally realized what the issue is.

For stereo channels, line 31 https://github.com/winstondu/Agora-RecordFrame-Bug/blob/e2269cd6bc05958441181faada72470d92aca8e3/SpeechRecognizer-iOS/MediaWorker.mm#L31

should have been

size_t size = audioFrame.bytesPerSample * audioFrame.samples * audioFrame.channels