Audio Support

[mrtnkhl]

There is currently no audio support planned until reverse engineering and interfacing with the digital bus is fully working. Creating this Issue as a placeholder for possible future work and to inform others who might be looking for audio (and video) support.

Per @DaSchaef the audio (and video signals) are modulated onto the wire in an analog format. It is suggested but has not been verified through testing that connecting a headphone amplifier via a coupling capacitor to the bus would yield audio signals. Likewise, connecting a line-out to the bus could allow sending audio signals to the (door) station. It is important to understand that analogue audio (and video) signals are only present after the digital bus command has been sent to the respective station to transmit audio (or audio and video) on the wires that also transport the digital bus signals.

See https://gdoor-org.github.io/documentation/protocol.html "Request Audio", "Request Video" for more details on the respective bus messages.

Since is (unfortunately) no digital audio (or video) signal are present, a combination of an ADC and DAC, most likely connecting via I2S to the ESP32 is needed to convert the analog signals to the digital domain. Using an ADC, DAC and a Raspberry Pi could be an option as well but this would require too many resources and power.

Per @DaSchaef, on the ESP32 side, there is additional work needed, mostly in software, like working with the DMA engine, timing Wifi transmissions correctly, etc.. This work can be summarized in the categories of coupling, digitizing, processing and streaming to an endpoint, e.g. via SIP or RTSP. If one wanted to have bidirectional, parallel communication, with the other (door) station some analog circuit magic to prevent audio feedback loops would be needed as well.

[mrtnkhl]

https://github.com/sikorapatryk/sip-call

Some previous work has been done using an ESP32 with an ADC and DAC as a SIP client, so maybe this can be of use here.

[mrtnkhl]

https://github.com/vvigilante/comelit-simplebus1

Another similar project but for Comelit, including the analog circuits interfacing with the ESP32 internal ADC and DAC for audio:

[mrtnkhl]

See this thread for a similar discussion for a Comelit system:
Elektroarzt/simplebus2-mqtt-bridge#3

[DaSchaef]

I successfully received audio via my computer:

• I just connected a cheap USB audio dongle to the bus (after the diode bridge, so that I do not need to care about polarity). (So that if something get's damaged it is only the cheap USB dongle)
• I used Audacity to record the audio from the USB dongle
• I send AUDIO_REQUEST via the GDoor v3.1 hardware adapter to my outdoor station
• I talked in front of the station

My voice was easily recorded on the computer.
So I can confirm that bus audio is just plain simple analog "line audio".

I would not recommend this setup so far, as for a real usage more protection and also a bit of filtering is needed.
In real world usage there would be multiple issues:

• The connected audio may disturb or "weaken" the bus signals
• The bus signals may damage the USB audio input

(both easily solvable)

After doing a lot of research:

• ESP video is a project in its own, we would solve problems which do not need to be solved.
  Even if we realize a solution (external ADCs etc), they will not be long lasting, as chip availability may change etc.
• ESP audio may be possible, but as video is not possible, let's just make one adapter.

My feeling is we should continue the audio and video route by just making plain simple adapters.
Kind of "bus to cinch analog video" and "bus to cinch analog audio".
These signals can be digitized with ~10€ USB Dongles and a Raspberry Pi, which itself can then act as VOIP phone,
image sender or whatever is needed.

[DaSchaef]

I have no Gira equipment here to test video. (if someone finds a cheap used video outdoor station 😉 , indoor is not needed for tests, we know how to request it 😅 )
So my raspberry pi solution / hat would be a "blind try" regarding video.

[mrtnkhl]

@DaSchaef do any of the Comelit circuits posted above look useful? It seems like they have bus-coupled audio ADC/DAC working and code for it as well on the ESP32 platform?

[mikburnz]

door opener works perfect. instead of 2 way audio it would be helpful to send one predefined message to the speaker. So i can advice the delivery to store items in front of my door f.e. Any suggestions if we can realize that?

[DaSchaef]

This should be doable as well,
I need to find a bit of time for it to make a ESP hardware. Currently I'm (very slowly) working on understanding ESP audio system. The most work is in the firmware/software, less in a hardware adaption.
I want to first understand what is needed and also doable in firmware/software so that the hardware does not lack a functionality needed.
(If we find someone with deep (!) ESP audio knowledge, better experience, it could speed this up. )

[DaSchaef]

I'm currently finishing a first try of hardware for audio support, yet I'm still struggling with firmware.
The goal is that the GDoor adapter behaves like an ESP audio development board, so I can re-use the ESP Voip code etc.

SD Card (and therefore playback) is more tricky, as with the current ESP board I am a bit limited by I/O pins.

• I need 4 people for the JLCPCB order of the first v4.0 boards to test.
• I do not plan/encourage to immediately send out the PCBs, as I want to first have at least a running board here,
  so that in worst case, I can modify the hardware before sending it out.

https://github.com/gdoor-org/gdoor/tree/audio

[DaSchaef]

Everyone who is interested and brave (it is very early alpha!),
please tell me, I'll create a private repo to exchange infos about price/payment etc.
Currently I'm still doing the PCB layout and final schematic checks.

[mrtnkhl]

I would be interested! ESPhome has audio support, e.g. to play out audio files. Maybe that would be an easier start?

[jschroeter]

I'm also interested!

[mikburnz]

I’m interested.

[HJoentgen]

Im also intereseted. I will ne installing the Gira system next week

[DaSchaef]

PCBs arrived today, now I need to test the firmware for audio support (which will also take a bit of time). I will keep you updated.

[DaSchaef]

I got a partly success, but it seems the ESP board is too weak (not enough memory) for the VOIP audio code.
Next I'll try with a WROVER module instead of WROOM, it can be patched to the existing PCB. ESP documentation is really bad 👎

https://www.amazon.de/dp/B09Z2GCCJZ

[DaSchaef]

With the new WROVER board I got a first voip connection (without being able to check audio so far). So at least the ESP code has no RAM memory issues anymore.

Testing VOIP via my FritzBox did not work, but it think the error is my FritzBox VOIP settings.
Next step will be to setup a local debug VOIP Server so that I can start to test audio and debug also with server side logs.

[mrtnkhl]

@DaSchaef super cool! Can you go into further details on what your added in the v4.0 revision of the board? How is the setup looking on the HW and SW side of things?

[DaSchaef]

Currently I'm using above shown custom PCB + the above ESP (Amazon Link) connected via jumper cables
(looks way more complicated than it is, it is a simple connection, no soldering or whatsoever).

I can now call my FritzBox from the ESP (GDoor Adapter). Audio is not working yet - this is now the next step.

Software is based on Esspressif ADF (https://github.com/espressif/esp-adf), I'm working in a private repo at the moment, as it is still very early/ugly, unstable and not working :) I'll merge it in the branch here once I'm sure that it is at least barely usable.

[DaSchaef]

For feature/HW changes in the v4.0 HW:

• I removed the variable RX threshold (because the Pin was needed for audio), only a fixed threshold is supported.
  This does not affect the user.
• I added a DC measurement of the bus voltage (so we can detect if bus is powered/connected).
• I added an ES8374 codec, which should make the audio in/out
• I added more on-board power filtering for the audio part
• I added a lot of debug pads related to the audio part

[mrtnkhl]

@DaSchaef very exciting! Thanks for the update! Looking forward to reading about the first successful 2-way audio call!

[DaSchaef]

I'm still struggling with audio, currently I only get silence in both directions - currently it looks like the S/W ist the reason and not the H/W. At least GDoor and VOIP seem to work in parallel. The ESP audio libs are a mess, but it is very slowly progressing - next step will be a small audio test script - not from the ESP libs, but from scratch - to have a minimal example to debug the missing audio. 😅

[DaSchaef]

Still no audio but I have an assumption:

The audio codec needs a MCLK generated by the ESP32.
The older ESP32 (incl. my boards here) can only output the precision MCLK on IO0,
which I designed in HW. But it seems the ESP boards use IO0 themselves for their "Bootloader Button",
therefore it can not be used although it is accessible on the pin header. 🤣

From my research the newer generation ESP32S3 is able to freely route high precision clock outputs to any of the GPIO.
So next try will be with yet another ESP board https://www.amazon.de/gp/product/B0CQNB74R9