New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Microphone array documentation #3

Open
doumdi opened this Issue Jul 21, 2016 · 20 comments

Comments

Projects
None yet
@doumdi

doumdi commented Jul 21, 2016

Could you please add information/documentation on how the microphone array works. Are the 8 channels available as "raw data" in the Raspberry Pi? What is the audio interface (I2S) ?

Thanks!

@eljuguetero

This comment has been minimized.

Collaborator

eljuguetero commented Jul 21, 2016

Hi,

We already implemented a raw 16 bits signed 16000 Hz raw interface over SPI
https://github.com/matrix-io/matrix-creator-hal/blob/master/demos/mic_demo.cpp

Better examples will be published the next week.

Best regards,

-- Andrés

@doumdi

This comment has been minimized.

doumdi commented Jul 21, 2016

Thanks for the quick answer. This will be perfect for voice applications! Will it be possible to have a higher sampling rate in the future?

@eljuguetero

This comment has been minimized.

Collaborator

eljuguetero commented Jul 21, 2016

Yes, we will improve the sampling rate. In your wish list, what sampling rate do you want?

@doumdi

This comment has been minimized.

doumdi commented Jul 21, 2016

48kHz would be great x 8 channels, 16bits. The intended application is audio beamforming. The sampling of the 8 microphones must be synchronous. How big is the audio buffer in the FPGA?

@doumdi

This comment has been minimized.

doumdi commented Jul 21, 2016

It seems the SPI communication is clocked at 25MHz. 8 microphones @ 48kHz, 16 bits = 6.144Mbps. That should fit.

@eljuguetero

This comment has been minimized.

Collaborator

eljuguetero commented Jul 21, 2016

2kB right now, but we have enough BRAMs to increase to 8kB.

@doumdi

This comment has been minimized.

doumdi commented Jul 21, 2016

8kB should store approximately 10ms of audio @ 48kHz. This could work.

@MexXxo

This comment has been minimized.

MexXxo commented Sep 14, 2016

Will this mic array demo also be available in nodejs?

@buildog

This comment has been minimized.

buildog commented Oct 6, 2016

+1 for js documentation for the mic & IR TX/RX

@preygel

This comment has been minimized.

preygel commented Oct 30, 2016

Second the suggestion on a higher sampling rate. I'd love either 44.1 or 48kHz support.

[Or if not, then perhaps a point to the current implementation / any documentation or considerations that would be useful in implementing the higher sample rate oneself.]

@namanigandhi

This comment has been minimized.

namanigandhi commented Nov 25, 2016

How to enable 10ms of audio @ 48kHz, 8 channel recording. Do we need to set any settings on Raspberry Pi.

@ruffsl

This comment has been minimized.

ruffsl commented Jan 12, 2017

Any update on this?
I too would like to achieve x8 channels @ 48kHz, 16bits
to use in audio beamforming for sound source localization and tracking.

@datascienceteam01

This comment has been minimized.

datascienceteam01 commented Mar 29, 2017

Is there a way to use the Alexa demo you guys created to simply stream the audio coming from the array, through wifi, to another system and for other purposes?

@paul-ml

This comment has been minimized.

paul-ml commented Apr 6, 2018

what will be the output of each channel separately?

@datascienceteam01

This comment has been minimized.

datascienceteam01 commented Apr 6, 2018

As was said earlier, people want to use the mic array to build custom beamforming experiments.

Matrix Creator was a disappointment, it was sold as a dev & test lab kit and it wasn't, rather a rushed & pretentious FPGA toy with a bunch of sensors crammed in for which a vague SDK and poor documentation was provided, mainly focused on accommodating the specific demo applications Matrix showcased to sell this. That is exactly what a dev board is not supposed to be. On top of that, terrible support: just look at how old this ticket is, after some social engineering I was able to reach them and I got this: "we're going to launch another product soon with x,y,z features which should accommodate your requirements." Anyone with capital can go to a chinese factory and commission 10000 FPGAs with lots o' sensors, but selling boards makes it your job to release a capable SDK, properly document and support for it on every angle, neither things Matrix did. Otherwise we'd all be making money stamping our band on chinese electronics.

@ruffsl

This comment has been minimized.

ruffsl commented Apr 6, 2018

As was said earlier, people want to use the mic array to build custom beamforming experiments.

This is very much what our lab tried to do with first the matrix creator and then again with the matrix voice, but had to drop entirely due to lack of any official released support and documentation from matrix.io for proper linux audio. If we have to bitbang the SPI bus to get raw ADC samples from the synchronized mic arrays ourselves using some odd sudo os layer in C, and flash the FPGA to ascertain better sampling rates than the course 16K, I'd rather just go with a small usb mic array like this one:

UMA-8 USB mic array

Multichannel USB microphone array for voice command
Onboard DSP for beamforming/ noise reduction / echo cancellation / de-reverd / Direction of Arrival (DOA) / Voice Activaty Dectection (VAD)
High quality MEMS from Knowles SPH1668LM4H
Simple USB control API for Direction of Arrival
Flexible Firmware (DSP mode @2ch beamformed, RAW mode @8ch audio from MEMS)
Step by step app notes for Google SDK, Alexa-Pi, Cortana, Siri and IBM Watson.
Resolution / Sample rate | 24bit @ 11/16/32/44.1/48 kHz
Linux Alsa 2.0 compliant

https://www.minidsp.com/products/usb-audio-interface/uma-8-microphone-array

@kdpatino

This comment has been minimized.

kdpatino commented Apr 6, 2018

Hello @ruffsl and @datascienceteam01

Thank you for your valuable feedback, It helps us to adjust our priorities and keep growing.

Here are answers to some of your concerns:

1- MATRIX Devices as a Linux device: We just released our first version of the MATRIXIO Kernel Modules . Resolution/Sample Rate: 16bit@8 to 96 KHz . https://github.com/matrix-io/matrixio-kernel-modules/releases

If you want to test it, you could run the next python example: https://github.com/matrix-io/matrixio-kernel-modules/blob/master/misc/pyaudio_test.py or use the alsa-utilities (arecord)

screenshot from 2018-04-06 15-51-25

2- Kernel Unification: Due to the kernel modules has been released, we made several adjustments in our software, in order to be compatible. Now we are finishing it, and the next week we will release this new version of our software. e.g: matrix-io/matrix-creator-hal#65

3- Support: @datascienceteam01 your welcome to join our community. there you could check some other projects and communicate with us easier. @ruffsl thank you to join our community and post there. I just want to invite you to keep post there any concern, doubt or issue. As I said is easier for us to answer there.

4- Documentation: We know that it is one of the things that we need to improve, you are right. We are making some efforts in order to improve it, but as you report these efforts had not been enough.

Thank You.

Best regards,

Kevin Patino and MATRIX Labs Team.

@datascienceteam01

This comment has been minimized.

datascienceteam01 commented Apr 6, 2018

@kdpatino We appreciate you coming in to provide support.
My team and I will take a look into the pointers you've given. I would however would like to note that I previously responded to this thread surprised of activity on a one year old open ticket that never took attention; during that time our team has moved to another platform as we had critical deadlines to meet. We felt disappointed and I'd now like to remind you that despite you referring to yourselves as a "community" in those keen "open source" terms, these are boards we paid for, and for which we got zero support when it was critically necessary. This being said, I do personally own one of your boards and I'll give every available tool you commented on a proper testing time.

Thank you for taking the time reviewing this.

@datascienceteam01

This comment has been minimized.

datascienceteam01 commented Apr 7, 2018

@kdpatino I've checked the .py you posted, it doesn't seem any different to what we already know. Please check @ruffsl post for an example of a mic array with a solid SDK to illustrate the framework features we were expecting of a board like "Creator". Notice @ruffsl has no affiliation, but we've both come to the same conclusions after experiencing the same shortcomings.

@stoneWeb

This comment has been minimized.

stoneWeb commented Jul 10, 2018

matrix-creator and matrix-voice How to implement DOA function?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment