Two way audio

[pergolafabio]

Hi @AlexxIT

Just a question, but is there a possibility to have two way audio enabled?

Thnx in advance

[danielducu]

I would definitely pay for that :)

[pergolafabio]

Yes, latest android app (beta) supports now microphone and speaker

[rbray89]

Technically this *may" be feasible, but it would likely depend on the method of backend audio. Either ONVIF 2-way would be assumed, or something like SIP would have to be used. Currently Home assistant is not aware of the audio sink, so it would have to be configured entirely in the component.

[pergolafabio]

Mine cameras are based on ISAPI, to start twoway audio, i need to send 2 commands (no onvif)
Synology is also using the same method for my hikvision door camera

would be a great feature, so we can chat with the HA app :-)

[rbray89]

WebRTC is also limited in supported codecs, which is why audio on some cameras doesn't work. This won't be an easy change to get it to work on devices that support 2way audio.

[pergolafabio]

Is it worth a try? Would be great!!
Mine audio codec is ulaw, it's the default audio protocol, most common used... One way audio already works now with current card...
But I have no idea how to enable the mic :-) and to send the ISAPi command from the card to active it on the stream...

[pergolafabio]

I have a trace how Synology is doing it, they just send 2 command to active the two-way audio

[AlexxIT]

RTSP is spec for camera stream translation. There is no spec for two way audio. Each company use their own tech. Usual it's some kind of P2P.

[pergolafabio]

yeah, mine does work with sending 2 commands, do you think its easy to integrate?

[rbray89]

@danielducu I don't think you understand the difficulty here.

1. vendors all do this differently, and it's not documented, so everything has to be reverse engineered.

2. webRTC only uses a few different audio codecs, so they'd almost certainly have to transcode things on the server.

[nanosonde]

If we rely on standards-based cameras, it could work though.

ONVIF Profile T: Chapter 8.12 Audio Output streaming: https://www.onvif.org/wp-content/uploads/2018/09/ONVIF_Profile_T_Specification_v1-0.pdf
More practical details: https://www.happytimesoft.com/knowledge/audio-back-channel.html
So basically a third stream towards the camera (besides video and audio from camera) is negotiated in the SDP of the server.
The audio out stream then just flows in the opposite direction of the other two RTP streams (sendonly vs. recvonly).

Video doorbell cameras mostly rely on two-way audio negotiated via SIP.
It did a PoC implemenation for Homebridge here: Sunoo/homebridge-camera-ffmpeg#1253
Ok, this is not related to WebRTC directly.

Solutions for propriertary vendor solutions could be doable. Example neolink for Reolink cameras: thirtythreeforty/neolink#268

Concerning ISAPI I only found this: https://www.hikvision.com/content/dam/hikvision/vn/webinar/Thang3_Hikvision_Tich_Hop_He_Thong_Overview-of-3rd-Party-Integration.pdf

[pergolafabio]

I have the Hikvision, and also all ISAPI documentations, so if i can help
i also know the ISAPI urls that should be used to activcate twoway audio

[mattkerrison]

The Amcrest AD410 Doorbell supports ONVIF Profile T for 2-way audio.

The Dahua integration for Home Assistant hasn't been able to get it working yet but the dream would be WebRTC with 2 way audio, could finally get rid of the garbage Amcrest app

[AlexxIT]

Do you know software for any OS, that can use ONVIF Profile T for 2 way audio connection?

[conorlap]

@AlexxIT
tinyCam on Android can do 2 way audio with these settings:

I found this out via Reddit: https://www.reddit.com/r/BlueIris/comments/lpkt8z/achieved_local_doorbell_with_2_way_audio/

[AlexxIT]

Seems like support backchannel in RTSP is not a big problem. Now I need to find some chip camera with this function in a local store.

[AlexxIT]

I'm wrote little app for test camera answer about backchannel. Can you test it?
https://github.com/AlexxIT/rtsplog

[pergolafabio]

That's a test for onvif? I have some foscam C1 cameras that have two way audio, maybe I can test too

[AlexxIT]

That test for any RTSP camera with 2 way audio. I'm already tested Sonoff camera and one Hikvision doorbell without success about backchannel in answer.

[pergolafabio]

Indeed ,I have hikvision too, for hikvision you need to send an ISAPI command to activate two way audio, I have the commands , also a wireshark trace from my Synology surveillance centre, where two-way audio is supported

[AlexxIT]

As I can see in this document - application just need to send right header with RTSP request. Can you see it in trace?

[pergolafabio]

No , Synology uses ISAPI on hikvision doorbell cameras , not rtsp , also onvif not supported, it's not the correct profile...
I also have foscam cameras, I can test later your tool

[mattkerrison]

I'm wrote little app for test camera answer about backchannel. Can you test it?

https://github.com/AlexxIT/rtsplog

Will try tomorrow when I have access to the AD410 camera

[pergolafabio]

Hey @AlexxIT , these 2 commands are fired with digest auth, thats what happens when i press the MIC button on the synology software....

[conorlap]

I'm wrote little app for test camera answer about backchannel. Can you test it? https://github.com/AlexxIT/rtsplog

I can test in the next day or two, how do I execute the rtsplog_amd64 file on Debian?

[AlexxIT]

@pergolafabio For now I don't have time for support some proprietary technology.

RTSP backchannel looks like nice standard. Which are not hard to support. But I can't find any cameras that support it yet.

[pergolafabio]

@pergolafabio For now I don't have time for support some proprietary technology.

RTSP backchannel looks like nice standard. Which are not hard to support. But I can't find any cameras that support it yet.

Ok, np...

And what about onvif?

[AlexxIT]

As I can understand, ONVIF just helps application to get right RTSP link. Plus some PTZ commands. I could be wrong.

[AlexxIT]

No. I don't have any camera that support ONVIF. Hope Dahua IPC-K42/22 will be first. But I can't order it without backchannel confirmation.

[pergolafabio]

Going to try again later, I changed my rtsp port , going to change back to 554

[AlexxIT]

Well. Good news. I just bought Dahua IPC-K42 (wi-fi version). And it has RTSP backchannel.
I'll try to add support in the next big update. I can't promise it will be soon.

[pergolafabio]

Nice to hear, maybe my foscams will work too :-)

[AlexxIT]

@mattkerrison you can check 2-way audio support:

https://community.home-assistant.io/t/realtime-camera-streaming-without-any-delay-webrtc/258216/420?u=alexxit

https://github.com/AlexxIT/go2rtc

[pergolafabio]

Nice, gonna check foscam too

[pergolafabio]

was testing this go2rtc, when i open web interface , i can access the cameras and view then in the addon web-ui
i added some rtsp links , works fine, all local , on http
https is for only avaible from outside, using nginx , i also added for test port forward 8554 and 1984 , but then i dont see the stream ... will this not work with nginx?

[AlexxIT]

Oh. You shouldn't forward RTSP port outside. Also you shouldn't forward API port, because this is insecure. Anyone can see your streams without auth...
Read docs carefully.

[pergolafabio]

yeah, it was just for testing, already closed
the problem is that i cant access my HA instance on https locally , only external
local i can only access http

[AlexxIT]

This is your setup problem. I have HTTPS inside and outside. This is fixed by Hairpin NAT or local DNS on router...

[pergolafabio]

ok, will have a look next time
propoably my foscams were not supported anyway, i already posted those logs before in this issue thread

[AlexxIT]

Also microphone will work on localhost. You can run go2rtc on your PC/laptop for tests.

Maybe some browsers has disable flag for this restriction. I don't know.

[pergolafabio]

seems defender is blocking the binary :-)

[pergolafabio]

Trojan:Win32/Trickbot!ml :-)

[pergolafabio]

ok, with binary it works, tested some rtsp , but no "mic" buton is present, so probably out of luck
or should i see the mic button?

[AlexxIT]

False virus detection because packer:
upx/upx#337
upx/upx#437

There is no warnings for unpacked version. But it is 15MB+.
I'll think what can do with this. Thanks for the point.

And stop generate so many separate messages.

[JxxIT]

Is there any update on this? Cause i really like to see this feature 😀.

[felipecrs]

I think this can finally become a thing now that the integration leverages go2rtc. :)

[danielducu]

I think this can finally become a thing now that the integration leverages go2rtc. :)

Not much, you can only do it with Onvif profile T cameras so not much people will benefit

[AlexxIT]

It is important that go2rtc has a core to support 2 way audio.

I will try to add support to the HomeKit cameras. There are a lot of those cameras. And many of them have 2 way audio. And the protocol is quite clear. Only the audio codec there is very bad. Thank to Apple.

[danielducu]

It is important that go2rtc has a core to support 2 way audio.

I will try to add support to the HomeKit cameras. There are a lot of those cameras. And many of them have 2 way audio. And the protocol is quite clear. Only the audio codec there is very bad. Thank to Apple.

Thanks a lot for your work man! don't get me wrong.. I'm a happy go2rtc user hehe ... this has come a long way so please, keep up the good work hehe !

[danielducu]

It is important that go2rtc has a core to support 2 way audio.

I will try to add support to the HomeKit cameras. There are a lot of those cameras. And many of them have 2 way audio. And the protocol is quite clear. Only the audio codec there is very bad. Thank to Apple.

You're awesome! :)

[felipecrs]

I think this can finally become a thing now that the integration leverages go2rtc. :)

Not much, you can only do it with Onvif profile T cameras so not much people will benefit

That's already enough. I hope support for profile T gets someday added to OpenIPC thus anyone with any camera can benefit.

[nanosonde]

Hi there!

Maybe you find my new project useful: https://github.com/nanosonde/sip2rtsp

It is docker container that uses baresip, pulseaudio and gstreamer RTSP server to provide a solution that connects to a SIP peer (audio only) and a video camera to present an ONVIF profile T video doorbell with two-way audio support.

There is also a block diagram which shows how the components interact.

[pergolafabio]

AlexxIT/go2rtc@5846cbd

thnx!!

[barrelltitor]

Is two way audio possible with eufy pan tilt 2k?