Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support I2S-ADCs/microphones #70

Open
renne opened this issue Mar 9, 2018 · 22 comments
Open

Support I2S-ADCs/microphones #70

renne opened this issue Mar 9, 2018 · 22 comments

Comments

@renne
Copy link

renne commented Mar 9, 2018

Hi,

you do a great job bringing I2S-audio to the ESPs! 👍

Please, support the I2S-input for I2S-ADCs and I2S-microphones like SPH0645LM4H.

@earlephilhower
Copy link
Owner

Actually, the Arduino-ESP8266 core team added I2S to the ESP, I just packaged codecs for use.

Adding input support requires changes in the arduino core. It's not rocket science, but it's not something I've dedicated any thought to.

What kinds of uses would you have for this? I've been thinking of grabbing a Google AIY box (deeply discounted @ Microcenter) and repurposing the MEMS microphone, but come up blank with ideas as to what could be done. 40KB of usable RAM and slow SPI writes to a small onboard flash feel limiting.

@renne
Copy link
Author

renne commented Mar 9, 2018

My idea is a multi-purpose MQTT sensor device with the ESP8266 for temperature, humidity, pressure, air quality, microwave motion detection, brightness, color temperature and a RGBWW LED-driver. Adding an I2S microphone and an I2S amplifier would allow bidirectional audio-streaming with a server for e.g. voice-control and audio-notifications.

@earlephilhower
Copy link
Owner

For Pi day, Microcenter is selling the AIY board for $3.14. I couldn't resist at that price. It turns out it has a stereo I2S microphone hat in addition to the MAX 1-channel DAC. https://www.invensense.com/wp-content/uploads/2015/02/ICS-43432-data-sheet-v1.3.pdf

There's also a project on GitHub recording from I2S: https://github.com/joextodd/listener

@renne
Copy link
Author

renne commented Mar 11, 2018

Lucky you!
In Germany we have to pay 30,- EUR. That's 240,- EUR for 8 rooms + 280,- EUR for the 8 RPis.

@earlephilhower
Copy link
Owner

Check the latest PR I submitted. It's got some work to do, but if you've got things wired up already I'd like to hear how it goes. My wire wrap tool has gone missing and I've got to find it before I can wire up that cheap stereo mike...

@h3ndrik
Copy link
Contributor

h3ndrik commented Aug 18, 2018

still interested in audio input support?

i'm trying to implement this but i'm not sure how to hook this into esp8266audio so this will actually become useful. Also this is neither a AudioGenerator nor a AudioFileSource. Do i need to inherit from something like that for some use cases?

https://github.com/h3ndrik/ESP8266Audio/tree/AudioInputI2S

File is called AudioInputI2S

h3ndrik added a commit to h3ndrik/ESP8266Audio that referenced this issue Aug 20, 2018
@earlephilhower
Copy link
Owner

That's a good start, @h3ndrik! Apologies in advance as I've found most of my free hacking time stuck on improving the Arduino core and not the audio bits.

I actually rewrote the I2S subsystem and figured out the (darn near undocumented) 8266 DMA engine to get input working, but never did anything with it.

I'm trying to figure out how this fits in the chain, logically. I think it could be an AudioGenerator, just one with a null input source. If I add a AudioOutputUlaw class, for example as compression, or an AudioOutputSocket class, for transmission, and an AudioOutputCallback class for the user's own handling of blocks, it may just fit into the framework with less mess.

Does that seem like it makes sense? Any other ideas?

@h3ndrik
Copy link
Contributor

h3ndrik commented Aug 20, 2018

uh. just filed the PR before i read this.
I could reformat this to inherit from AudioGenerator. For now I implemented nearly the same methods.

Is there already an AudioOutputUlaw class or a network output somewhere?

@earlephilhower
Copy link
Owner

Thanks for the updated PR. There is no AudioOutput(U,A)law, yet, but it's one thing I'll need to add as it could easily give 2:1 compression with minimal CPU and quality loss, which is important for the 8266!

I'm really swamped this week an next with travel and meetings, so I don't know if I will be able to look at it in depth and merge until the 1st week of September, but it's definitely on the to-do list!

I also got a note about doing the same thing for bluetooth a2dp (actually I guess you could do an input and an output mode) on the 32, but haven't looked at it at all. I think the same framework could work as this, which would help prove the AudioGenerator() interface is a usable model.

@h3ndrik
Copy link
Contributor

h3ndrik commented Aug 22, 2018

Alright. No need to hurry, I'll always have my own forks and branches.

I'm working on RTP (U,A)Law-streaming in the meantime.

@Morgaul
Copy link

Morgaul commented May 24, 2019

Hi. It will be very usefull to have AudioOutputOPUSclass for voice (VoIP) applications...

@dmtulsa
Copy link

dmtulsa commented Jul 17, 2019

I'm doing a project that requires real time bidirectional audio but it doesn't have to be real high quality and one way video 30fps esp32 >> android / ios and would like to use I2s . Just thought you may want too know what others are looking to do. I know its hard work I'm a retired software engineer. But I;m too old and too far behind to to develop anymore. Not even sure the esp32 can handle that.

@jamesboblak-school
Copy link

I'm interested in getting voice audio into an I2S microphone/ESP32 and sending it realtime to another ESP32/I2S amp. I have built a transmitter and receiver with SPH0645 and MAX98357A. They are connecting to each other via wifi, but I still can't get audio out of the speaker. Did this tread continue elsewhere? I'm concerned that it may simply not be possible. Thank you for your incredible work here!

@earlephilhower
Copy link
Owner

@jamesboblak for your use, a simple walkie-talkie, this library really isn't going to help since it's all about decompression.

I'd just use the basic Espressif I2S calls for that...on a UDP packet receive, just write it to the I2S buffer. And for I2S in, just resend it via UDP when you get notified that one buffer is available.

@frankcohen
Copy link

frankcohen commented Mar 19, 2021 via email

@jamesboblak-school
Copy link

jamesboblak-school commented Mar 19, 2021 via email

@frankcohen
Copy link

frankcohen commented Mar 19, 2021 via email

@jamesboblak-school
Copy link

jamesboblak-school commented Mar 19, 2021 via email

@frankcohen
Copy link

frankcohen commented Mar 19, 2021 via email

@jamesboblak-school
Copy link

jamesboblak-school commented Mar 19, 2021 via email

@jamesboblak-school
Copy link

jamesboblak-school commented Mar 21, 2021 via email

@jamesboblak-school
Copy link

jamesboblak-school commented Mar 21, 2021 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants