ALSA alsa_snd_pcm_hw_params() on Ubuntu 20.04 started returning an unexpected -ENOMEM

[philburk]

Theo Veenker reported a bug on the PA list.
We may need a workaround in PortAudio.

---

Under Ubuntu 20.04 (clean out of the box) I'm encountering a problem with
Pa_IsFormatSupported(). The attached test program demonstrates the problem. Under Linux
Mint 18 (which is based on Ubuntu 16.04) the program works fine, with both
v190600_20161030 and v190700_20210307. Under Ubuntu 20.04 however it reports a failure at
hostapi/alsa/pa_linux_alsa.c line 1818 (or a few lines up for v190600_20161030).

The problem also occurs in Audacity according to its debug output.

The problematic part is this (in pa_linux_alsa.c):

     int ret = 0;
     if( ( ret = alsa_snd_pcm_hw_params( pcm, hwParams ) ) < 0 )
     {
         if( -EINVAL == ret )
         {
             /* Don't know what to return here */
             result = paBadIODeviceCombination;
             goto error;
         }
         else if( -EBUSY == ret )
         {
             result = paDeviceUnavailable;
             PA_DEBUG(( "%s: Device is busy\n", __FUNCTION__ ));
         }
         else
         {
             result = paUnanticipatedHostError;     line 1815
         }

         ENSURE_( ret, result );                    line 1818
     }

Apparently alsa_snd_pcm_hw_params() returns -12 (which is -ENOMEM). No memory seems
unlikely here. If I comment out lines 1815 and 1818 the program runs fine. So the device
can play audio just fine, but it is just not possible to check whether a particular
format/config is valid.

[philburk]

Attached is a test program provided by Theo.

test1.cpp.txt

[philburk]

On the ALSA repo, takaswie did some excellent research and came up with a couple workarounds.

Please try the workaround here and comment on that ALSA Issue page:

alsa-project/alsa-lib#125 (comment)

[kleinerm]

Fwiw, testing on Ubuntu 20.04.3-LTS with manually installed Linux 5.13 fixes the problem for me.

Another way that worked for me to work around the bug on older kernels was to change the allowable
maximum buffer size, e.g., to 64 MB, via

echo 67108864 > /sys/module/snd_pcm/parameters/max_alloc_per_card

I encountered the bug a year ago and my analysis can be found here, fwiw:
kleinerm/Psychtoolbox-3@0f89230

It suggests the trouble started with Linux 5.6 when checks for maximum buffer size were introduced,
so Ubuntu 20.04.2-LTS or Ubuntu 20.10 were the first affected Ubuntu versions with Linux 5.8.

Unfortunately I forgot to submit a suitable bug report or fix at that time, once i had my workaround in
place for my own software. I think it is great that Portaudio is now hosted on a popular platform like
GitHub, as it makes issue reporting or pull requests just a little tad less overhead for lazy people like me.

Thanks for your continued work on this great software.

[xyzzy42]

The patch to Portaudio from the ALSA bug report thread fixes the problem. While this appear to only be a significant problem from kernel 5.6 to 5.12, it would be nice Portaudio worked correctly on these kernels too.

I think the patch to Portaudio is not really a workaround but fixes a flaw in Portaudio. For actual audio use, Portaudio specifies a reasonable buffer size in the hwparams requested from ALSA. But for testing format support, no buffer size is specified, resulting in hwparams with an unreasonably large buffer that fails on allocation. In kernel 5.13 a large, but not as large, buffer will be used and this (probably) does not fail. But Portaudio should not leave the size unspecified, so that a normal buffer size is used, as it does for actual audio use. Otherwise format checking will always specify hwparams that are unlike the ones used for Portaudio's, and anyone else's, real use and these will hit obscure edge cases that are not normally triggered.

Also, it wastefully causes the kernel to allocate a largest possible buffer that will not even be used.

[RossBencina]

Ok, so the suggested patch from alsa-project/alsa-lib#125 (comment) is:

$ diff -u src/hostapi/alsa/pa_linux_alsa.c.orig src/hostapi/alsa/pa_linux_alsa.c 
--- src/hostapi/alsa/pa_linux_alsa.c.orig	2021-03-13 17:54:54.520167673 +0900
+++ src/hostapi/alsa/pa_linux_alsa.c	2021-03-13 17:55:52.304748119 +0900
@@ -1754,6 +1754,7 @@
     unsigned int numHostChannels;
     PaSampleFormat hostFormat;
     snd_pcm_hw_params_t *hwParams;
+    snd_pcm_uframes_t limitation;
     alsa_snd_pcm_hw_params_alloca( &hwParams );
 
     if( !parameters->hostApiSpecificStreamInfo )
@@ -1788,6 +1789,19 @@
     /* Some specific hardware (reported: Audio8 DJ) can fail with assertion during this step. */
     ENSURE_( alsa_snd_pcm_hw_params_set_format( pcm, hwParams, Pa2AlsaFormat( hostFormat ) ), paUnanticipatedHostError );
 
+    /*
+     * Intel HDA driver doesn't set PCM rule to limit maximum size of buffer.
+     * This brings request for too large buffer size and causes memory allocation
+     * error in ALSA PCM core, at least Linux kernel 5.8. As a workaround, limit
+     * buffer size up to 10 msec.
+     */
+    limitation = alsa_snd_pcm_format_size(SND_PCM_FORMAT_S16_LE, 48000 * 2 / 100);
+    if( alsa_snd_pcm_hw_params_set_buffer_size_near( pcm, hwParams, limitation ) < 0 )
+    {
+        result = paBufferTooBig;
+        goto error;
+    }
+
     {
         /* It happens that this call fails because the device is busy */
         int ret = 0;

We should merge this I think.

[xyzzy42]

I think basic idea is correct. Implementation is flawed, assumes 48000 kHz two channel. Better would be to use snd_pcm_hw_params_set_buffer_time_near() to suggest a ~100 ms buffer at the sample rate and channel count being used. Alternatively one could use params->suggestedLatency as the buffer time instead of 100 ms.

Or use a fixed buffer size of 512, which is what is done in GropeDevice().