AAudio glitches / crackles on S8, S9

[Tim-Kindberg]

I'm developing an audio recording app for an external USB device. I'm using AAudio with the intention of switching to Oboe later if possible (and hence to support older devices). I'm using callbacks, and I'm careful not to perform bocking operations in the handler - I create a thread periodically (1 sec interval) to compress the data acquired so far.

I'm using a Samsung S8 (Android 8.0, API 26) and S9 (9, 28).

I find, as some others have reported, glitches/crackles. They are very brief and seem to occur sparsely and randomly. I note that others have tried changing the buffer capacity. I've increased it by various amounts up to 0.5secs at a 48000 sample rate with no improvement. Irrespective of that, the frame numbers being passed in the callback seem to be 64 or 960 or a few variations e.g. 17 then 47 instead of 64 (I haven't checked exhaustively).

My initialisation code is:

AAudioStreamBuilder_setDeviceId(recordingBuilder.get(), mSourceDeviceId);
AAudioStreamBuilder_setDirection(recordingBuilder.get(), AAUDIO_DIRECTION_INPUT);
AAudioStreamBuilder_setPerformanceMode(recordingBuilder.get(), AAUDIO_PERFORMANCE_MODE_NONE);
AAudioStreamBuilder_setSharingMode(recordingBuilder.get(), AAUDIO_SHARING_MODE_EXCLUSIVE);
AAudioStreamBuilder_setFormat(recordingBuilder.get(), AAUDIO_FORMAT_PCM_FLOAT);
AAudioStreamBuilder_setSampleRate(recordingBuilder.get(), sampleRate); // 48000
AAudioStreamBuilder_setChannelCount(recordingBuilder.get(), kChannelCountStereo); // 2
AAudioStreamBuilder_setBufferCapacityInFrames( recordingBuilder.get(), 4800 );     // 100ms at 480000 per sec
AAudioStreamBuilder_setDataCallback(recordingBuilder.get(), ::recordingDataCallback, this);

Any thoughts on how I might investigate further?

[philburk]

I've increased [capacity] by various amounts up to 0.5secs at a 48000 sample rate with no improvement.

Input streams tend to run near empty. So glitches due to overflows at the application level are unlikely. The glitches are probably occurring at a lower level.

What USB devices have you tried? Do they all glitch?

Are you doing output at the same time?

Do you see glitches on other phones?

I create a thread periodically (1 sec interval)

Spawning a thread from an audio callback is a blocking operation. Try spawning the thread once then put a FIFO between the callback and the compression thread.

the frame numbers being passed in the callback seem to be 64 or 960 or a few variations e.g. 17 then 47

Odd callback sizes can be a sign of resampling by AudioFlinger.

Try recording with OboeTester. Does it work any better?

[philburk]

Irrespective of that, the frame numbers being passed in the callback seem to be 64 or 960 or a few variations e.g. 17 then 47 instead of 64

You might try setting a specific framesPerDataCallback. That will prevent the seemingly random callback sizes but may add few msec of latency.

[Tim-Kindberg]

Hi Phil,

Many thanks for getting back to me. I've been trying a few things but I'm still getting the crackle and I get it with OboeTester, too. To respond to your points:

• I have tried only the USB hardware I'm writing custom software for (I'm going to get hold of another type of USB device); but I've also tried the internal mic and I can hear crackling in that recording too
• I'm not doing output (playback) at the same time; I'm encoding the data and writing it to a file
• Yes, it happens on a Samsung S8 & S9, and a Moto G5+
• I've changed the threading logic so that I create a thread once that waits in a loop on a condition variable; the callback is never blocked
• I also tried buffering many seconds of audio before I do any processing - same result
• I'm not concerned about the variable callback sizes - unloess they might be indicative of an underlying problem?

I tried a commercial app that uses its own driver (Audio Evolution Mobile Studio pro) and I'm not hearing the crackle on that.

[atneya]

@philburk could this be related to the noise we were seeing earlier? I know we were not using USB, but perhaps an underlying issue with certain devices?

[philburk]

@atneya - The noise we were seeing was hiss, like static. I think this is about glitches, which are caused by gaps in the stream.

[philburk]

@Tim-Kindberg - We have found a problem on some devices when:

1. the USB is running at a rate != 48000,
2. and the DSP is doing resampling,
3. and the stream is MMAP (not Legacy).

Resampling in the DSP causes odd timing that is not accounted for by the MMAP Timing Model.

When you record the stream it will say "MMAP" or "Legacy" in OboeTester, right above the frames_read counter.

Does that match what you are seeing?

[Tim-Kindberg]

Hi Phil,

Under Test Input I'm getting "start failed with -895" with OboeTester (which I thought I'd tried before) - API unspecified, Srate 48000, Format unspec, Perf. NONE, external USB device selected.

However, I can tell you that I was exclusively trying a sample rate of 48000. And I'm pretty sure the device doesn't use MMAP. Part of the delay in opening the device with AAudio seemed to be where something tried to find out if it was MMAP and timed out.

FYI, I have started using a commercial libusb-based driver https://extreamsd.com/. With it, I can make clean recordings at 44.1kHz. But not at 48k. Go figure...

Tim

[philburk]

Thanks Tim. To recap, you are running your app at 48000 and PerfMode NONE.

Part of the delay in opening the device with AAudio seemed to be where something tried to find out if it was MMAP and timed out.

What did you see in the logs that suggest that?
The PerfMode NONE should prevent AAudio from trying to open MMAP.

I have tried only the USB hardware I'm writing custom software for

Does your device support 44100 as an option? The odd callback sizes suggest that something is doing resampling somewhere. Also the fact that xstreamsd "can make clean recordings at 44.1kHz" also suggest the USB is running at 44100 while the app is at 48000.

Test Input I'm getting "start failed with -895"

-895 is invalid state. Which device was that on? You can get that if you try to do a state transition that is unnecessary, like calling START twice in a row. There might be a leaked stream in OboeTester. If you swipe away the OboeTester app so it is closed (or reset the phone) do you still get an -895?

Some things to try:

• In your app, try not setting the capacity. That will allow AudioFlinger to choose an optimal capacity.
• In OboeTester, try "Record and Play" test. It will record the last 10 or 20 seconds of audio. If you record a glitch then click "Share". Then you can email yourself a WAV file and attach it here. By analyzing the glitch, we may be able to figure out its origin.
• In OboeTester, "Record and Play", try setting API to OpenSL ES.

I've also tried the internal mic and I can hear crackling in that recording too

Do they sound similar. Maybe record both. If you get the same glitches through the mic then let's concentrate on the mic. That eliminates your custom USB device as a cause.

[philburk]

@Tim-Kindberg Any news? We can close this if you are ready.

[paulorrfilho]

Hi, having a similar issue on a Samsung S8. Currently trying to port my drum app from iOS to Android.
I have been studying the RhythmGame example with my own mp3 files. Tried many different buffer sizes and Audacity export settings. Cracking noises occur every time and it is inconsistent as sometimes I get few seconds with no noises. PerformanceMode is set to LowLatency.
Any other ideas of what might be happening?

[sbanzhaf]

Hi, I am also seeing the issue of glitches/crackles on S8, S9 and a variety of other galaxy phones. I'm using oboe to create an AAudio output stream at 48000 Hz in LowLatency mode. Additionally, I also observe the issue using the provided hello-oboe application on these phones as well.

Referencing the final post #286 as of this writing, changing the performance mode to None seems to alleviate the issue, but this is not an acceptable solution for me as low latency audio is vital to the functionality of our applications. The glitches/crackling do not seem to occur on all phones, as I have not observed the glitches/crackling on the OnePlus 5 or OnePlus 6 with the exact same code/applications running.

It's been several months since the last post on this. Are there any updates on this issue?

[dturner]

The original issue from @Tim-Kindberg was to do with input streams, whereas the later posts from @paulorrfilho and @sbanzhaf were about output streams so we have a bit of issue contamination here. I will just address the issue of pops/crackles on the input stream here. For output streams please file a separate issue (although I did do a quick test using a Galaxy S9 and Android 10 and was not able to reproduce as long as the buffer size was increased to 192, the default in OboeTester was 96).

Firstly, here's a test for attempting to reproduce the "pops/crackles in input stream" issue:

• Load OboeTester
• Tap "Echo Input To Output"
• Make a single pitch "aaaaaah" sound into the microphone and turn up the volume until you can hear it being echoed through the device's speaker, but not so loud that you get feedback.
• Continue making an "aaaaah" sound and listen to the output from the speaker
• Verify that there are no clicks/pops or other artifacts in the output audio

I tried running this test with a Samsung Galaxy S9 running Android 10 (build QPA1.190711.020.G965FXXSBETH2) and was not able to hear any pops/crackles. For this reason I'm closing as unable to reproduce.

[Tim-Kindberg]

Hi Don, FWIW, I was using external USB hardware, and that's what gave me the glitches (with AAudio and Oboe) - didn't try it with an internal mic. We eventually used a driver based on libusb, which worked fine. As it happens, recently there have been reports that people with Samsung S20 Ultras (& possibly other high-end devices) *don't* hear glitches when utilising this same hardware with different apps via native drivers - and that our app is conflicting with that use. Maybe a performance artefact? Just an observation. Can I also observe that we wanted to avoid the Android scheme to automatically launch our specific app when this USB device is plugged in, by removing the device resource entry. But when we do that, and programatically ask for permission to use the device when the app is launched, Android asks *every time* the app is launched - even though some variants have a tickbox to remember that choice. Is there a workaround for that? Cheers, Tim

On 17/09/2020 15:21, Don Turner wrote: The original issue from @Tim-Kindberg <https://github.com/Tim-Kindberg> was to do with /input/ streams, whereas the later posts from @paulorrfilho <https://github.com/paulorrfilho> and @sbanzhaf <https://github.com/sbanzhaf> were about /output/ streams so we have a bit of issue contamination here. I will just address the issue of pops/crackles on the input stream here. For output streams please file a separate issue (although I did do a quick test using a Galaxy S9 and Android 10 and was not able to reproduce as long as the buffer size was increased to 192, the default in OboeTester was 96). Firstly, here's a test for attempting to reproduce the "pops/crackles in input stream" issue: * Load OboeTester * Tap "Echo Input To Output" * Make a single pitch "aaaaaah" sound into the microphone and turn up the volume until you can hear it being echoed through the device's speaker, but not so loud that you get feedback. * Continue making an "aaaaah" sound and listen to the output from the speaker * Verify that there are no clicks/pops or other artifacts in the output audio I tried running this test with a Samsung Galaxy S9 running Android 10 (build QPA1.190711.020.G965FXXSBETH2) and was not able to hear any pops/crackles. For this reason I'm closing as unable to reproduce. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#564 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJEE5CW5JA2BJNF46LVEODSGILQBANCNFSM4H4UFEEQ>.

-- Tim Kindberg matter II media matter2media.com grapevine.ink ididntbuy.org vorb.me nthscreen.tv Pervasive Media Studio Bristol UK +44 (0)7954 582814

[dturner]

@Tim-Kindberg Thanks for the extra info, I had missed the part about it being from a USB audio device. I was able to reproduce this behaviour using a NI Komplete Audio 6 USB audio interface using the following method:

• Connect a USB audio device to a Samsung Galaxy S9
• Load the OboeTester app: https://github.com/google/oboe/tree/master/apps/OboeTester
• Tap "Echo Input To Output"
• Play a sound into the USB audio device. A clean sine wave is probably best as it's easy to hear any audio artifacts.

Expected result: Sound is played without pops and crackles on the USB audio device.

Actual result: Sound contains pops and crackles on the USB audio device.

I have filed this bug internally with Samsung: b/168810849 and will update here if there's any progress.

[dturner]

@Tim-Kindberg As far as the USB dialog is concerned, since this isn't an Oboe specific issue please could you file an issue here and send the bug id to me at donturner@google.com and I'll direct it accordingly.