AudioCapture throws DllNotFoundException "asound" in Moby Container running on Raspbian GNU/Linux 10 (buster)

[nicolasiten]

I'm trying to do an AudioCapture in a Moby Container (Azure IoT Edge) running on Raspbian GNU/Linux 10 (buster).

The following packages are installed (in the Container and on the host):

• alsa-utils
• libasound2
• libasound2-dev

Below is the Exception I get:

System.DllNotFoundException: Unable to load shared library 'asound' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libasound: cannot open shared object file: No such file or directory
   at Microsoft.Psi.Audio.LinuxAudioInterop.Open(Void** handle, String name, Int32 capture, Int32 mode)
   at Microsoft.Psi.Audio.LinuxAudioInterop.Open(String name, Mode mode, Int32 rate, Int32 channels, Format format, Access access)
   at Microsoft.Psi.Audio.AudioCapture.Start(Action`1 notifyCompletionTime)
   at Microsoft.Psi.Executive.PipelineElement.<Activate>b__46_0()
   at Microsoft.Psi.Scheduling.Scheduler.ExecuteAndRelease(SynchronizationLock synchronizationObject, Action action, SchedulerContext context)
   --- End of inner exception stack trace ---
   at Microsoft.Psi.Pipeline.ThrowIfError()
   at Microsoft.Psi.Pipeline.Run(ReplayDescriptor descriptor)

Here's the code I'm using:

            using (var pipeline = Pipeline.Create())
            {
                // Windows Version
                //var audioCapture = new AudioCapture(pipeline, new AudioCaptureConfiguration
                //{
                //    DropOutOfOrderPackets = true,
                //    OutputFormat = WaveFormat.Create(WaveFormatTag.WAVE_FORMAT_PCM, _rate, 16, _channels, 2, 88200),
                //});
                var audioCapture = new AudioCapture(pipeline, new AudioCaptureConfiguration
                {
                    Format = WaveFormat.Create(WaveFormatTag.WAVE_FORMAT_PCM, _rate, 16, _channels, 2, 88200)
                });

                audioCapture.Do(async (buffer, envelope) => await ProcessAudioMessage(buffer, envelope));
                pipeline.Run();
            }

I've tested the solution on a Windows machine, there it works without any problem.

[AshleyF]

Hi Nicolas,

The Linux versions of the AudioCapture and AudioPlayer \psi components are very thin wrappers over the ALSA APIs and depend on the asound shared object. The symptom of a missing asound lib seems consistent with something being wrong with your libasound2 installation (apt install libasound2-dev).

An approach might be to test your audio hardware and asound installation outside of \psi first. You may record and playback audio with the arecord and aplay command line utilities. For example, to record and playback a 10 second test clip:

arecord -f S16_LE -d 10 -r 16000 -D hw:1,0 test.wav
aplay test.wav

You can list available capture (arecord -L) and playback (aplay -L) hardware to determine device names. You may want to experiment with sample rates and formats to ensure that your settings are correct (though, you'd be getting a different error if that were the root issue here).

Once in \psi, the AudioCapture and AudioPlayer components each take configuration details, as I see you're using, including the DeviceName (default "plughw:0,0") and Format (default 16KHz, 1 channel, 16-bit PCM).

Hopefully this is helpful. Please let us know how this goes!

[nicolasiten]

Hi Ashley,

Thanks a lot for your feedback!

I've tried to run the arecord and aplay on the host directly and that seems to work without any issues.

Also the microphone is listed under the available capture hardware list.

I could imagine that there's a package missing in my Container. I've added a few more and now I've installed the following:

• alsa-utils
• libasound2
• libasound2-dev
• libasound2-data
• libasound2-plugins

Could you think of any other package I could try? Or do you have any other idea what could go wrong?

[AshleyF]

I'm at a loss, honestly. I don't see how arecord and aplay could work without asound installed. Our code is merely importing functions from asound.

Is the libasound.so in your /lib or /usr/lib?

find /usr/lib -name *asound*

Or, if not, is its location listed in /etc/ld.so.conf or seen by ldconfig -p | grep asound?

[nicolasiten]

It looks like libasound2-dev wasn't installed properly. However after fixing that, I get the following Exception:

ALSA lib pcm_hw.c:1713:(_snd_pcm_hw_open) Invalid value for card

Unhandled Exception: System.AggregateException: One or more errors occurred. (Pipeline 'default' was terminated because of one or more unexpected errors (Open failed.)) ---> System.AggregateException: Pipeline 'default' was terminated because of one or more unexpected errors (Open failed.) ---> System.ArgumentException: Open failed.
   at Microsoft.Psi.Audio.LinuxAudioInterop.Open(String name, Mode mode, Int32 rate, Int32 channels, Format format, Access access)
   at Microsoft.Psi.Audio.AudioCapture.Start(Action`1 notifyCompletionTime)
   at Microsoft.Psi.Executive.PipelineElement.<Activate>b__46_0()
   at Microsoft.Psi.Scheduling.Scheduler.ExecuteAndRelease(SynchronizationLock synchronizationObject, Action action, SchedulerContext context)
   --- End of inner exception stack trace ---
   at Microsoft.Psi.Pipeline.ThrowIfError()
   at Microsoft.Psi.Pipeline.Run(ReplayDescriptor descriptor)
   at AudioAnalyzer.Program.Init() in /app/Program.cs:line 93
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at AudioAnalyzer.Program.Main(String[] args) in /app/Program.cs:line 36

I guess that indicates that asound is unable to find the Device. When I run arecord -l I get the following Device:

**** List of CAPTURE Hardware Devices ****
card 1: Device [USB PnP Sound Device], device 0: USB Audio [USB Audio]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

In my understanding the default DeviceName should be correct: plughw:0,0. However since the card is 1 I also tried the following: plughw:1,0. With both I get the same error.

Do you see any other reason why the ALSA lib pcm_hw.c:1713:(_snd_pcm_hw_open) Invalid value for card could come up?

[nicolasiten]

Seems like I didn't set the Container Options correctly that's why Alsa probably wasn't able to find the Device. However after fixing that I still get the following not very detailed Exception:

Unhandled Exception: System.AggregateException: One or more errors occurred. (Pipeline 'default' was terminated because of one or more unexpected errors (Open failed.)) ---> System.AggregateException: Pipeline 'default' was terminated because of one or more unexpected errors (Open failed.) ---> System.ArgumentException: Open failed.
   at Microsoft.Psi.Audio.LinuxAudioInterop.Open(String name, Mode mode, Int32 rate, Int32 channels, Format format, Access access)
   at Microsoft.Psi.Audio.AudioCapture.Start(Action`1 notifyCompletionTime)
   at Microsoft.Psi.Executive.PipelineElement.<Activate>b__46_0()
   at Microsoft.Psi.Scheduling.Scheduler.ExecuteAndRelease(SynchronizationLock synchronizationObject, Action action, SchedulerContext context)
   --- End of inner exception stack trace ---
   at Microsoft.Psi.Pipeline.ThrowIfError()
   at Microsoft.Psi.Pipeline.Run(ReplayDescriptor descriptor)

[AshleyF]

We're down to a configuration problem I believe. What name and other parameters work when using arecord?

The failure to open could be due to the name, mode, rate, channels, format, ... I'd try to get it working with arecord first and then use those settings in \psi. Maybe less opaque errors.

BTW, I believe that arecord -L (with capital L) will display the names (e.g hw:0,0, hw:0,1) as well. Example:

$ aplay -L
default:CARD=CK804
    NVidia CK804, NVidia CK804
    Default Audio Device
front:CARD=CK804,DEV=0                 # hw:0,0
    NVidia CK804, NVidia CK804
    Front speakers
surround40:CARD=CK804,DEV=0            # hw:0,1
    NVidia CK804, NVidia CK804
    4.0 Surround output to Front and Rear speakers

[nicolasiten]

The following works to record a Wave file:
arecord -f S16_LE -d 10 -r 16000 -D hw:1,0 test.wav
I use the following configuration in my code:
Format = WaveFormat.Create(WaveFormatTag.WAVE_FORMAT_PCM, 44100, 16, 1, 2, 88200)

I don't get the names with arecord -L:

arecord -L
null
    Discard all samples (playback) or generate zero samples (capture)
default:CARD=Device
    USB PnP Sound Device, USB Audio
    Default Audio Device
sysdefault:CARD=Device
    USB PnP Sound Device, USB Audio
    Default Audio Device
front:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    Front speakers
surround21:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    2.1 Surround output to Front and Subwoofer speakers
surround40:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    4.0 Surround output to Front and Rear speakers
surround41:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    4.1 Surround output to Front, Rear and Subwoofer speakers
surround50:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    5.0 Surround output to Front, Center and Rear speakers
surround51:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    5.1 Surround output to Front, Center, Rear and Subwoofer speakers
surround71:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    7.1 Surround output to Front, Center, Side, Rear and Woofer speakers
iec958:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    IEC958 (S/PDIF) Digital Audio Output
dmix:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    Direct sample mixing device
dsnoop:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    Direct sample snooping device
hw:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    Direct hardware device without any conversions
plughw:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    Hardware device with all software conversions

However I guess if arecord with 1,0 works I have to use plughw:1,0 as DeviceName.
This leaves me with the following Exception:

Unhandled Exception: System.ArgumentException: Read recovery failed.
   at Microsoft.Psi.Audio.AudioCapture.<Start>b__15_0()
   at System.Threading.Thread.ThreadMain_ThreadStart()
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()

[AshleyF]

To summarize, the following works on your hardware:

arecord -f S16_LE -d 10 -r 16000 -D hw:1,0 test.wav

And the following throws with "Read recovery failed." in \psi:

Format = WaveFormat.Create(WaveFormatTag.WAVE_FORMAT_PCM, 44100, 16, 1, 2, 88200)
\\ BTW, Format = WaveFormat.Create16BitPcm(44100, 1) is equivalent

I'm curious, did arecord show a message such as "Warning: rate is not accurate (requested = 16000Hz, got = 44100Hz)"? Is that why 44100 is used in the \psi config? Does arecord -r 44100 ... work?

Also does Name = "hw:1,0" fail in \psi and/or does -D plughw:1,0 work with arecord? Basically, trying the exact same settings across arecord and \psi.

Assuming none of this works, the next thing I can think of is to clone and build \psi locally and debug to see what the actual err number is from the ALSA APIs here. I've added a work item for myself to add these error numbers to the exceptions we throw throughout to make this easier in the future.

[nicolasiten]

The following works with arecord:
arecord -f S16_LE -d 10 -r 44100 -D hw:1,0 test.wav
And the following fails in psi:

DeviceName = "plughw:1,0",
Format = WaveFormat.Create16BitPcm(44100, 1)

Yes exactly I get the warning regarding the 16000Hz that's why I'm using 44100.
And yes I guess I'm referring to the same device.
Also I've analyzed the Header of the wav file I generated with arecord and it looks as expected:

Sample Rate: 44100
Avg Bytes Per Sec: 88200
Bits Per Sample: 16
Num Channels: 1

I'll try to clone the library and check the error number.

[nicolasiten]

I cloned the library and added Logs in the LinuxAudioInterop class.
The Read Method call (line 414) returns the following code:
-4574459702921920256
The Recover Method call (line 417) returns the following code:
-256
Hope that helps.

[AshleyF]

Humm... I'm not able to make sense of those. I wish I could repro on my end. My next step would be to try to get a simple C program to run using just the ALSA APIs and then figure out what is wrong with the \psi wrapper. This could very well be a bug in \psi and I feel like I'm having you do my job for me at this point :)

If possible, could you try building and running this sample app?

https://github.com/ashleyF/linux-audio

[nicolasiten]

I'm able to run your sample app without any error:

./audioTest
Bit-width: 16
Capturing...
Buf 0: 2 5 3 4 1 6 ...
Buf 1: 13 13 11 12 17 6 ...
Buf 2: 9 9 5 11 9 8 ...
.......

To make sure the problem is not due to an incorrect Device mapping in my container I've built a little sample Net Core Console App to run it on the host directly.
Here's the Code of the sample app:

    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Hello World!");

            using (var pipeline = Pipeline.Create())
            {
                var audioCapture = new AudioCapture(pipeline, new AudioCaptureConfiguration
                {
                    DeviceName = "plughw:1,0",
                    Format = WaveFormat.Create16BitPcm(44100, 1)
                });

                audioCapture.Do(async (buffer, envelope) => await ProcessAudioMessage(buffer, envelope));
                pipeline.Run();
            }
        }

        private static async Task ProcessAudioMessage(AudioBuffer buffer, Envelope envelope)
        {
            Console.WriteLine($"Received Audio Bytes {buffer.Length}");
        }
    }

However I get the same output (added some more logs):

Hello World!
plughw:1,0
44100
1
S16LE
Read error number -4574459702921920256
Recover error number -256
Unhandled exception. System.ArgumentException: Read recovery failed.
   at Microsoft.Psi.Audio.AudioCapture.<Start>b__15_0() in C:\Users\nicolas.iten\Downloads\psi-master\psi-master\Sources\Audio\Microsoft.Psi.Audio.Linux\AudioCapture.cs:line 138
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
Aborted

[nicolasiten]

Just tested on a different Linux Device (Debian, arm7 32bit).
It still doesn't work but I get a different error number:

./Audio
Hello World!
plughw:0,0
44100
1
S16LE
Read error number 4294967552
Unhandled exception. System.ArgumentException: Read failed.
   at Microsoft.Psi.Audio.AudioCapture.<Start>b__15_0() in C:\Users\nicolas.iten\Downloads\psi-master\psi-master\Sources\Audio\Microsoft.Psi.Audio.Linux\AudioCapture.cs:line 138
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
Aborted

[AshleyF]

Can I see the code producing this "Read error number" message? It seems like some fixednum conversion or sign extending or some such problem with reporting the error code. In fact, a positive error number from snd_pcm_readi() doesn't even make sense. Only negative numbers are errors. Positive numbers are supposed to be the number of bytes read (but 40GB?!).

None of these error codes are making any sense (e.g. snd_strerror(...) returns "Unknown error ...").

Also, does the sample C app work on this new hardware?

[nicolasiten]

It's the same code I posted 2 comments ago. I did all my tests with that one.
arecord works on both devices and if I remember correctly so does your sample C app.
Do you have any idea what else we could test or install?
I'm starting to get out of ideas and I need to get this running as soon as possible.

[AshleyF]

I just added a .NET project to the Linux Audio test repo. This is not using \psi, but is using the same p/invokes directly to the ALSA APIs. There must be a difference between the calls being made by the working C app and either this .NET sample (if it fails) or \psi. Examining the code though, I'm failing to see it...

[nicolasiten]

I tested the .NET sample and this one returns the following output and error code:

ALSA Test App
Opened
Params
Params initialized
Params set access
Params set format
Params set rate
Params set channels
Set params
Free params
Prepare handle
Error: Read failed. -4574459702921920384

We even tested with different microphones/soundcards but always the same result.. It works with arecord and the c app but not with the C# apps.

[AshleyF]

Looking closely at the C (working) vs. C# (broken) samples, the only difference I see is that the C version is actually not freeing the snd_pcm_hw_params_t while the C# version is. Also, one uses "plughw:0,0" and the other "plughw:1,0" but I assume you changed that.

Perhaps, could you try uncommenting this line in the C code. If that fails, then perhaps you could try commenting out this line in the C# sample and/or this line in the \psi component.

I'd be interested to hear the result! I guess the theory would be that something under the covers may be holding a reference?

[nicolasiten]

Yes, I noticed that as the only difference between the C# and the C sample. And I tried to comment that line out in C# but sadly that didn't change anything. I even checked if all the parameters match, which is the case.
Also I made sure to use the same DeviceName for both apps of course.
Since I lost a lot of time with that issue and I had to come up with a solution I implemented a workaround. If the app is running on Windows I'm using the Psi.Audio Library to read and process the data. However if it's running on a Linux system I'm now using arecord to get the data and Psi.Audio only to process it.
So currently I'm not further investigating the problem and I came up with a solution.
If there's something else I can test out to help fixing the problem let me know!

[AshleyF]

I'm glad you have somewhat of a workaround; capturing outside of \psi with arecord. The one last thing I might ask is, could you tell me the hardware and Linux distro version (Raspbian, correct? On Raspberry Pi [version?], external microphone?), etc. that you're using and I can perhaps try to repro this.

Thanks so much for pushing on this. We've added ALSA error output to our component (next release) and will hopefully figure this bug out at well.

[nicolasiten]

Thank's for putting in so much of effort to try to help me!
We're using the following distro:

Distributor ID: Raspbian
Release:        10
Codename:       buster

And the following hardware:

Raspberry Pi 3 Model B Plus Rev 1.3
Microphone: Adafruit Mini USB Microphone

I will keep myself updated regarding this and I'm looking forward to use the psi library in future projects. If you need further information or if I can help in any way you can reach out to me.

[AshleyF]

Okay I got a Raspberry Pi and I think I've figured it out. It's that the P/Invokes of the ALSA APIs are assuming 64-bit while Raspbian is 32-bit ARM.

We'll get the fix into the next release. I updated the little sample test app here. To patch this in \psi you could change these two P/Invokes to take uint (instead of ulong) and to return int. That will then only work on 32-bit. We'll be fixing this in a more complete way in the next release.

Thanks again for bringing this bug to our attention!

[AshleyF]

This is now fixed in the latest release. We're closing this issue now. Thanks once again for bringing this to our attention.