ESP32 support

[BahtiSlo]

Hi,recently i make esp32 based media player true esphome in home assistant which work perfect in home assistant. It plays radio stations and local media files but is not visible in music assistant. Is there any limitation which esp32 bord is supported by music assistant?

[OzGav]

At this point it needs to be running the squeeze lite firmware

[BahtiSlo]

Thank you! I will check squeeze lite, but i think that i need different esp32 board (wroom is not supported) if i know correctly.

[YahyaAlfayad]

I am having a similar situation. All my speakers at home are esphome media players with ESP32 board on them. Are you planning to bring back the support for them?

[OzGav]

I don’t believe it is possible to use a combined voice assistant and media player

[AquaWolf]

Ok I mean that's fine is there a esp home config for the muse luxe I could start with which is working well with MA? I only need the button control for HA. The Voice Assistant is not important for me.
Edit: Found the firmware on https://esphome.io/projects/?type=media
I'll try that one 👍
Edit 2: Ok i tried that firmware TTS from HA directly is working but music (yt music) from MA is not playing the speaker is just shortly cracking and not playing the music. I also tried tunein and a raw url to a internet radio station.
I'm getting this in the MA logs:
ERROR (MainThread) [music_assistant] Error doing task: Task exception was never retrieved RuntimeError: anext(): asynchronous generator is already running

[OzGav]

Have you selected the mp3 stream in the player options?

[AquaWolf]

Yes the force mp3 was active. Meanwhile I installed squeeze lite but the esphome firmware would give me more flexibility.
Should I set up the log level to get more information?

[OzGav]

No. If the native player works (i.e. via squeezelite) then we aren't putting any resources into trying to make HA media players work at this time.

[jherby2k]

I think we got you covered now. I was wondering if there is also a demand to have a native ESPHome player provider in MA to send music to these players ? I'm just wondering if anybody sees a usecase for that or we call this request as closed/fulfilled.

So i'm in the process or replacing all my Google Home devices (voice assistants) with ESP32-based "assist" ones. I can't use the other firmwares and still use them as voice assistants too. So yeah - ideally we need a high-quality ESPHome option. Since these things also don't work directly with Spotify, i'm even more reliant on Music Assistant for a music streaming solution.

[OzGav]

I am not sure that a voice assistant combined with a high-quality media player exists?

[marcelveldt]

At this time no, its just too much to handle for the ESP chip but I'm sure better results will come once dedicated voice processing chips get combined with ESPHome/ESP32.

For now most folks just use ESPHome for the voice processing (and of course all other goodies ESPHome can offer) and then use dedicated media players for good quality media playback, the combination is not (yet) possible.

[ignacio82]

would it be possible to run two esp32 boards with the same speaker? One running espohome with home assistant, one running snapcast? Is this a bad idea?

I just got music assistant to play on my esphome smart speaker by forcing mp3 encoding but the audio is too choppy to be usable.

[OzGav]

https://www.diyaudio.com/community/threads/two-amps-into-a-speaker.299420/

[Hedda]

At this time no, its just too much to handle for the ESP chip but I'm sure better results will come once dedicated voice processing chips get combined with ESPHome/ESP32.

would it be possible to run two esp32 boards with the same speaker? One running espohome with home assistant, one running snapcast? Is this a bad idea?

@marcelveldt not sure how much you are involved in Nabu Casa's voice-assistant hardware projects but heard Paulus Schoutsen just reveal on Home Assistant's ESPHome Summer Release Party on YouTube that Nabu Casa's ESPHome developers are working on a new open-source hardware platform for their voice-assistant products that will based on ESP32-S3 in combination with a very powerful XMOS xCORE chip for audio processing.

https://www.youtube.com/watch?v=hbcGz3ZlUX4

I think similar xCORE chips from XMOS is by the way used in Amazon Alexa Voice Service (AVS) Development Kit(s) and Amazon Echo products:

• https://www.xmos.com/usb-multichannel-audio/
• https://www.xmos.com/develop/usb-multichannel-audio/
• https://www.xmos.com/xmos-delivers-first-amazon-alexa-voice-service-development-kit-with-linear-mic-array-for-far-field-voice-capture/
• https://developer.amazon.com/en-US/blogs/alexa/post/edde861e-65a4-4319-b5f1-422b0c626673/xmos-expands-device-types-supported-by-dev-kits-for-avs-with-a-far-field-linear-mic-array-based-reference-solutio
• https://developer.amazon.com/en-US/alexa/devices/alexa-built-in/development-resources/sdk
• https://github.com/xmos/vocalfusion-avs-setup

...and while I am unsure of it I would suspect that Google Nest / Google Home smart speaker series also contain XMOS xCORE chip?

• https://www.xmos.com/fully-offloaded-giving-smart-tvs-the-voice-power-they-deserve/
• https://www.xmos.com/making-smart-speakers-feel-at-home/

Anyway, since Nabu Casa's designs it said to be open-source hardware and XMOS integration will probably be added to the ESPHome's Media Player Components (and Microphone Components) I for one am hoping that could and will be extended to other type of audio output and audio input solutions and not only for voice-assistant.

• https://esphome.io/#media-player-components
• https://esphome.io/#microphone-components
• https://esphome.io/#speaker-components

Personally I would also love to see inexpensive speakerless network-streamer player/receiver hardware without microphones but only with with AUX-out that can connect to any of your existing amplifiers or speakers with built-in amplifiers in order to replace products like Chromecast Audio and Amazon Echo Input / Echo Link Amp, (e.i. devices with no on-board speakers that must be connected to external speakers for audio output

That is, I am sure that not everyone only wants "smart speakers" with voice-assistant and that instead many would be also happy to have network streamers/players without microphone which only purpose is to receive and output highest quality audio possible from Music Assistant to your "dumb" speakers.

I for one still have loads of Chromecast Audio audio-only receivers connected to various models of different speaker system in each room used to achieve multi-room music playback on a budget (because could not afford Sonos speakers in all rooms).

So even if though Nabu Casa's hardware will initially primarly be designed for "Home Assistant Satellite" (also known as "Wyoming Satellite") for voice-assistant appliances, such open-source hardware it just like the ESPHome firmware does have a lot of potential for different use cases:

• https://github.com/rhasspy/wyoming-satellite
  • https://github.com/synesthesiam/homeassistant-satellite
• https://www.home-assistant.io/integrations/wyoming/

I do however think that such a solution does need its own non-propriatory audio-only streaming protocol for high-quality music streams?

PS: Just as cool would it be with network streamer receiver hardware with AUX-input and ADC to get music from analog audio source, see:

• https://github.com/orgs/music-assistant/discussions/2343

[Hedda]

FYI, the newly released ”ReSpeaker Lite" (ReSpeaker Lite board only) and “ReSpeaker Lite Voice Assistant Kit” is a new voice-kit hardware board from Seeed Studio that features combining ESP32 with XMOS XU-316 xCORE AI (a MCU and DSP chipset) for advanced voice/audio processing:

https://community.home-assistant.io/t/respeaker-lite-voice-assistant-kit-seeed-studio-voicekit-combining-xmos-xu-316-and-esp32-s3/756944

Note that they offer different models on the same webpage, with one being a 2-Mic Array board model that combines ESP32-S3 ESPHome support + a XMOS XU-316 chip for advanced audio processing, and a second model that is a DIY-varient that is only a 2-Mic Array board model with just the XMOS XU-316 chip that you can use with your own compute solution (thus you need to add your own MCU board or a SBC/computer such as a the Raspberry Pi) and connect it via I2S or USB.

The former full kit solution sound similar to the hardware specification mentioned for the upcoming voice-kit development platform that Nabu Casa members said that they are working on, or? …though looks like the ReSpeaker Lite Voice Assistant Kit is missing expansion ports and if so will not allow for additional hardware addons?

[Hedda]

WOOT! Stumbled on this new "voice-kit" GitHub repository where ESPHome developers are developing new or improved components for I2S audio (XMOS) support and a new native media player with support for FLAC, MP3, etc. for the upcoming voice-kit hardware from Nabu Casa:

• https://github.com/esphome/voice-kit/
  • https://github.com/esphome/voice-kit/tree/dev/esphome/components
    • https://github.com/esphome/voice-kit/tree/dev/esphome/components/nabu
    • https://github.com/esphome/voice-kit/tree/dev/esphome/components/i2s_audio
    • https://github.com/esphome/voice-kit/tree/dev/esphome/components/media_player
    • https://github.com/esphome/voice-kit/tree/dev/esphome/components/voice_kit
    • https://github.com/esphome/voice-kit/tree/dev/esphome/components/audio_audio_processor

They already added features and functions or improvements/enhancements to ESPHome, such as:

• New: Nabu Media Player - new "nabu" media player from Nabu Casa running natively on ESP32
  • Music Assistant streams work (both mp3 and flac), but since it requires resampling, the audio quality isn't great
• New: Added support for FLAC files
• New: Added a proper WAV decoder (that parses WAV headers with LIST, INFO, etc. chunks.)
• New: Initial support for playing back local files
• New: Playback Control for the VoiceKit
• New: Added an is_paused condition for media players.
• New: Add Click to Converse to button
• New: LED animation
• New: Scripts for controlling LEDs
• New: Update Button Behaviour for the Voice kit
• New: Dial Volume Control
• New: Timer basic implementation
• New: Dial Volume Control
• New: Added HTTP(s) OTA updates
• New: Dial Volume Control
• New: Added Buttons for force ota update.
• New: Software Mute Switch
• Improvement: A basic resampler adjusts sample rates
• Improvement: Configurable output sample rate (for experimental 48kHz XMOS firmware)
• Improvement: The DAC mute state is read on boot
• Improvement: volume/mute control via the DAC (the wheel works for increasing/decreasing volume)
• Improvement: Logs what element failed if the pipeline breaks
• Improvement: Fails gracefully if the incoming stream can't be processed
• Improvement: Differentiate between user facing LED Ring and Internal LED ring
• Point external component to dev branch

They also have many TODO inline coments in the code there if anyone are interested in helping them:

https://github.com/search?q=repo%3Aesphome%2Fvoice-kit%20todo&type=code

Note! Be aware that there are many comments there to that most of the new stuff are not yet stable.

[Hedda]

FYI, also check out this new "ESPHome media proxy" feature for auto-converting of formats being adding upstream support to Home Assistant:

• ESPHome media proxy home-assistant/core#123254

plus related "MediaPlayerSupportedFormat" patches for ESPHome, ESPHome API, and new voice-kit components that is in development:

• Add MediaPlayerSupportedFormat esphome/aioesphomeapi#925

and

• Add MediaPlayerSupportedFormat esphome/home-assistant-voice-pe#55

Hopefully those will make it to Home Assistant 2024.9 and ESPHome 2024.9 which if so means this new feature could be made use of after that.

[Hedda]

Also check out the work @rwrozelle has started working on laying the groundwork for extending child components of Media Player in ESPHome (and Home Assistant) to allow ESPHome to be built with a much richer set of capabilities in the media_player. See:

• https://github.com/rwrozelle/audio-media-player
  • Extend Functionality of Media_Player esphome/aioesphomeapi#911
    • Extend Functionality of Media_Player esphome/esphome#7158
    • Extend Functionality of Media_Player Documentation esphome/esphome.io#4090
    • Extend Functionality of ESPHome Media Player home-assistant/core#122799