Add support for two-way audio for SIP based video doorbell #1355

longzheng · 2022-11-08T23:08:03Z

This PR incorporates @nanosonde's PR #1253 and includes various fixes I proposed to address some potential bugs with SIP clients like my 2N doorbell.

… same port for TX and RX (symmetric RTP)

For example my config was ``` "source": "-i rtsp://192.168.1.3/h264_stream", ``` and the FFmpeg command was executed as ``` ffmpeg.exe -i rtsp://192.168.1.3/h264_stream-hide_banner -protocol_whitelist pipe,udp,rtp,file,crypto... ``` which leads to an error since `rtsp://192.168.1.3/h264_stream-hide_banner` is not the correct stream URL.

Fixes issue with my 2N SIP doorbell not correctly responding to the INVITE if it receives an empty `tag` in the `to:` header. For example ``` to: "SIP doorbell client" <sip:192.168.1.3>;tag ``` would not work correctly whereas ``` to: "SIP doorbell client" <sip:192.168.1.3> ``` works

ryan99alero · 2022-11-12T20:43:03Z

I installed this PR today. I added the following for the camera config. Camera stream works but no audio nor the talk button at bottom. Going to dig a little more if there are some things I need to set inside the added new code. Not sure if I need to still check the Enable Audio checkbox. I think previously I didn't have.

    {
        "name": "Camera FFmpeg",
        "cameras": [
            {
                "name": "DoorBellSIP",
                "model": "AXIS",
                "serialNumber": "9238923J09LJ2",
                "firmwareRevision": "1.2",
                "doorbell": true,
                "unbridge": true,
                "videoConfig": {
                    "source": "-i http://username:password@192.168.29.55/axis-cgi/mjpg/video.cgi?",
                    "stillImageSource": "-re -stream_loop -1 -i http://username:password@192.168.29.55/axis-cgi/mjpg/video.cgi? -f alsa -ac 2 -ar 48000",
                    "debug": true,
                    "debugReturn": true
                },
                "sipConfig": {
                    "to": "sip:100@192.168.29.55", //IP of HomeBridgeServer
                    "from": "sip:100@192.168.29.72", //IP of Axis DoorBell
                    "debug": true,
                    "debugReturn": true
                }
            }
        ],
        "_bridge": {
            "username": "0E:AE:7B:DC:43:25",
            "port": 41080
        },
        "platform": "Camera-ffmpeg"
    }

mrMiimo · 2022-11-12T21:00:27Z

I installed this PR today. I added the following for the camera config. Camera stream works but no audio nor the talk button at bottom. Going to dig a little more if there are some things I need to set inside the added new code. Not sure if I need to still check the Enable Audio checkbox. I think previously I didn't have.

@ryan99alero how did you manage to install this pr?

longzheng · 2022-11-12T22:07:42Z

I installed this PR today. I added the following for the camera config. Camera stream works but no audio nor the talk button at bottom. Going to dig a little more if there are some things I need to set inside the added new code. Not sure if I need to still check the Enable Audio checkbox. I think previously I didn't have.

Yes you still need the "audio": true, since that determines if any of the audio logic is enabled. The SIP logic builds on top of that so it requires it.

ryan99alero · 2022-11-13T08:22:03Z

@mrMiimo Not sure how much you work with git. One of the other guys pointed out using "npm link" but I'm not sure if you can get to PR's with that or not. With Git you have to first clone the repository, Then fetch the PR by its ID, Then Checkout "IE step into the PR", Grab all the required packages for the build, Run the build command to prepare the package for install. Next I pack it up and then use the NPM command to install the compressed tgz file. You'll need to pay attention to the outputs of the commands and make sure they aren't failing. Below is the basic. This will work for this particular GitHub and particular PR.

Go into a direction you want to download the program. Often the home folder via.

cd ~

Then download and do all the things I listed above with commands below. Anyone feel free to point out how I'm doing this wrong and provide correct way. The "npm outdated -g --depth=1" command should hopefully come back empty. If not run it again after running the npm update command. If still returning a package it could be because it's a global package. Those you can fix with "npm install -g PackageName@latest". The area named PackageName is the name of the package that gets shown in result/s from npm outdated -g --depth=1 command. The @latest will install latest build of said package. Otherwise you can do @BuildNumber that is returned in the Wanted column. Hope I haven't confused you.

git clone https://github.com/Sunoo/homebridge-camera-ffmpeg.git
cd homebridge-camera-ffmpeg/
git fetch origin pull/1355/head:SIP
git checkout SIP
npm outdated -g --depth=1
npm update
npm audit fix
npm run build
npm pack

sudo npm install -g homebridge-camera-ffmpeg-3.1.4.tgz

ryan99alero · 2022-11-13T08:53:46Z

@longzheng > > I installed this PR today. I added the following for the camera config. Camera stream works but no audio nor the talk button at bottom. Going to dig a little more if there are some things I need to set inside the added new code. Not sure if I need to still check the Enable Audio checkbox. I think previously I didn't have.

Yes you still need the "audio": true, since that determines if any of the audio logic is enabled. The SIP logic builds on top of that so it requires it.

SIP not working but I didn't see any attempts via TCPDump. Haven't dug to much yet. I haven't installed yet but I assume I still need to install baresip and alsa loopback? I didn't see them in the nod_modules folder so they don't appear to be packaged with the project or listed in package.json.

longzheng · 2022-11-13T08:55:51Z

SIP not working but I didn't see any attempts via TCPDump. Haven't dug to much yet. I haven't installed yet but I assume I still need to install baresip and alsa loopback? I didn't see them in the nod_modules folder so they don't appear to be packaged with the project or listed in package.json.

Not infront of my computer but you shouldn't need any more dependencies. I think those are dependency of sip?

ryan99alero · 2022-11-13T20:28:58Z

@longzheng In log I get the errors below. I've tested with LinPhone for Peer to Peer and Axis Device works. Going to try between LinPhone and HomeBridge to get some additional logs. Next will be to mirror switch port for Axis and see what additional data I can gather. From my wireshark I see nothing going to or from ports 5060 or 5061. I'd imagine you'd see traffic on those ports on your working environment. Anyway you can post your whole homebridge config xml like I posted above. So I can make sure I'm not doing something wrong.

[11/13/2022, 8:21:31 PM] [Camera FFmpeg] [DoorBellSIP] [error] Output file #1 does not contain any stream
[11/13/2022, 8:21:31 PM] [Camera FFmpeg] [DoorBellSIP] [AVIOContext @ 0x55c66e3e1480] [verbose] Statistics: 0 bytes written, 0 seeks, 0 writeouts
[11/13/2022, 8:21:31 PM] [Camera FFmpeg] [DoorBellSIP] [AVIOContext @ 0x55c66e3ebb80] [verbose] Statistics: 101368 bytes read, 0 seeks
[11/13/2022, 8:21:31 PM] [Camera FFmpeg] [DoorBellSIP] FFmpeg exited with code: 1 and signal: null (Error)

longzheng · 2022-11-13T23:12:56Z

@ryan99alero My config is

        {
            "name": "Camera FFmpeg",
            "cameras": [
                {
                    "name": "Intercom test",
                    "doorbell": true,
                    "unbridge": true,
                    "videoConfig": {
                        "source": "-analyzeduration 0 -i rtsp://192.168.1.3/h264_stream",
                        "stillImageSource": "-i http://192.168.1.3/api/camera/snapshot?width=1280&height=960",
                        "vcodec": "copy",
                        "audio": true,
                        "debugReturn": true
                    },
                    "sipConfig": {
                        "to": "sip:192.168.1.3",
                        "from": "sip:127.0.0.1"
                    }
                }
            ],
            "platform": "Camera-ffmpeg",
            "porthttp": "5051",
            "_bridge": {
                "username": "0E:E6:4A:C1:93:81",
                "port": 36383
            }
        },

ryan99alero · 2022-11-14T03:32:07Z

I would expect to see something listing on port 5060 or 5061 for VOIP ports so when one pushes the button on the doorbell. Thats the port the homebridge has a port listener running. Can you show what your output is from sudo ss -tulpn of linux or windows netstat -an. I'd also be curious if you have a notification action set on your doorbell for when the button is pushed or if it's just set to dial a SIP. I'd imagine it has to be more than just SIP so you'd get a notification event plus get video feed. I originally had it working with your previous work around but it used baresip service that was running and listening for incoming connection to fire things off and kick of the alsa loopback. I did some reading and the kirm/sip.js module doesn't use baresip or alsa loopback. So it'd have to somehow be listening for incoming sip connections.

longzheng · 2022-11-14T03:41:29Z

I would expect to see something listing on port 5060 or 5061 for VOIP ports so when one pushes the button on the doorbell. Thats the port the homebridge has a port listener running. Can you show what your output is from sudo ss -tulpn of linux or windows netstat -an. I'd also be curious if you have a notification action set on your doorbell for when the button is pushed or if it's just set to dial a SIP. I'd imagine it has to be more than just SIP so you'd get a notification event plus get video feed. I originally had it working with your previous work around but it used baresip service that was running and listening for incoming connection to fire things off and kick of the alsa loopback. I did some reading and the kirm/sip.js module doesn't use baresip or alsa loopback. So it'd have to somehow be listening for incoming sip connections.

Sorry I think there's some confusion with what the functionality of the original PR is.

It does not run a SIP server/keep the port 5060/5061 open and wait for the doorbell to dial it, that was not the intention of the original issue either.

Instead, it only initiates a SIP call to the device/intercom when you open the camera view so you get two-way audio, so the microphone audio from your iOS device can be sent to the SIP intercom.

To trigger the actual doorbell, you need to use the existing homebridge-camera-ffmpeg doorbell functionality which is triggered by MQTT or HTTP (in my case my doorbell triggers via HTTP to Homebridge).

ryan99alero · 2022-11-14T04:24:52Z

@longzheng Ok I understand. I was just think multiple steps ahead. Not sure whats going on. It doesn't initiate any kinda outbound SIP call for me currently. Think I will wipe my VM and redo. This time I'm going to take snapshots along path so I can revert. I think I'll first try nanosonde initial fork when I had it working the first time on a different doorbell and if it works take a snapshot and revert back to clean build and try yours again. Only thing I can think of is my npm build while its compiling isn't building right or something.

ryan99alero · 2022-11-14T04:37:47Z

That or it's something to do with enabling audio support as thats when it craps out. Maybe something to do with the feed or kind of stream my doorbell sends since it throws an error: "Output file #1 does not contain any stream" Then it closes the stream out. So its likely this is my issue I need to resolve as its not getting to the SIP portion of the integration.

longzheng · 2022-11-14T05:18:39Z

That or it's something to do with enabling audio support as thats when it craps out. Maybe something to do with the feed or kind of stream my doorbell sends since it throws an error: "Output file #1 does not contain any stream" Then it closes the stream out. So its likely this is my issue I need to resolve as its not getting to the SIP portion of the integration.

Which device do you have?

ryan99alero · 2022-11-14T05:36:19Z

Which device do you have?

Currently testing against an Axis Q8105E but i also have an Axis a8207-ve. Currently installed is a DoorBird D101 and it's working via the HomeBridge DoorBird plugin. I could test against the door bird as it supports sip as well. I wish they'd implement webRTC for these things. Would be way easier.

nanosonde · 2022-11-14T07:41:49Z

"sipConfig": {
                        "to": "sip:192.168.1.3",
                        "from": "sip:127.0.0.1"
                    }

Mhmm. This looks suspicious. I do not see any user/extension part in the SIP URI.
On my doorbell I have this:

"sipConfig": {
                        "to": "sip:11@<ip_of_my_doorbell>",
                        "from": "sip:user1@<ip_of_homebridge_instance>"
                    }

Without the "11" as user it does not work as the doorbell is configured to use this extension when somebody presses the doorbell button.

ryan99alero · 2022-11-14T12:55:43Z

Without the "11" as user it does not work as the doorbell is configured to use this extension when somebody presses the doorbell button. I had extensions in mine see below and way above for my config section for this plugin. I tried both as 100 and one as 101 as well. I haven't tried 11 as in if the code is hard coded. I can make it all work with the LinPhone app or work on a desk phone going through either my local asterisk or through my works SBC and then to PBX. Going to try every version of URL schema the axis supports to see if I can find a url / combo that doesn't throw the "Output file #1 does not contain any stream" error.

"sipConfig": {
"to": "sip:100@192.168.29.55", //IP of HomeBridgeServer
"from": "sip:100@192.168.29.72", //IP of Axis DoorBell
"debug": true,
"debugReturn": true
}

longzheng · 2022-11-15T03:19:38Z

"sipConfig": {
                        "to": "sip:192.168.1.3",
                        "from": "sip:127.0.0.1"
                    }

Mhmm. This looks suspicious. I do not see any user/extension part in the SIP URI. On my doorbell I have this:

"sipConfig": {
                        "to": "sip:11@<ip_of_my_doorbell>",
                        "from": "sip:user1@<ip_of_homebridge_instance>"
                    }

Without the "11" as user it does not work as the doorbell is configured to use this extension when somebody presses the doorbell button.

I'm pretty sure for direct SIP the user/extension is not required. I can make a direct call to my Intercom with just sip:192.168.1.3.

mcm246 · 2023-02-02T05:25:17Z

@longzheng @Sunoo Are there any plans to merge this pull request into the master branch? It would be great to have it included as part of the official plugin, as I’m having a hard time getting it installed on my own.

Thank you for your work on this project!

longzheng · 2023-02-02T05:46:54Z

Sorry I haven't progressed any further with this branch and I won't be able to support it as I've moved from homebridge-camera-ffmpeg to Scrypted for my Homekit camera solution as it already has native ONVIF and SIP handling.

Sunoo · 2023-02-02T05:48:20Z

I can merge this, I just cannot test it, so I can't say for sure it'll work right.

mcm246 · 2023-02-02T13:15:19Z

Understood. I’m happy to test it with my doorbell setup once merged and report back. In the meantime, I may check out Scrypted’s SIP support as @longzheng suggests.

longzheng · 2023-02-02T23:13:05Z

Understood. I’m happy to test it with my doorbell setup once merged and report back. In the meantime, I may check out Scrypted’s SIP support as @longzheng suggests.

If you wanted to, you could install this PR directly without it being merged first

npm install -g Sunoo/homebridge-camera-ffmpeg#pull/1355/head

nanosonde · 2023-02-24T16:01:20Z

@longzheng
I have also moved to Scrypted. I have written SIP plugin for scrypted to achieve the same as here.

However, maybe you find my new project useful: https://github.com/nanosonde/sip2rtsp

It is docker container that uses baresip, pulseaudio and gstreamer RTSP server to provide a solution that connects to a SIP peer (audio only) and a video camera to present an ONVIF profile T video doorbell with two-way audio support.

There is also a block diagram which shows how the components interact.

So basically you can use the ONVIF plugin from scrypted and let the above docker container do the rest.

350d · 2023-08-01T21:36:03Z

@Sunoo Hello! Any plans to merge this PR and other in line? Thanks!

nanosonde and others added 12 commits January 15, 2022 12:47

First working two-way audio SIP call

6b5c3f7

Cleanup

d3d5f2e

Cleanup and create SipCall instance at startup

5f4dddd

Added RtpHelper to fix problem of having two ffmpeg instances use the…

3b9ce58

… same port for TX and RX (symmetric RTP)

Add configuration via config file and cleanup

e6aebaf

Cleanup

bfa6026

Cleanup and add possibility to specify SIP port for SIP client

180bdcf

Cleanup

0a506ba

Cleanup

5536895

Add failure reason logging

272af9b

longzheng mentioned this pull request Nov 8, 2022

Add support for two-way audio for SIP based video doorbell #1253

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for two-way audio for SIP based video doorbell #1355

Add support for two-way audio for SIP based video doorbell #1355

longzheng commented Nov 8, 2022

ryan99alero commented Nov 12, 2022

mrMiimo commented Nov 12, 2022 •

edited

longzheng commented Nov 12, 2022

ryan99alero commented Nov 13, 2022

ryan99alero commented Nov 13, 2022 •

edited

longzheng commented Nov 13, 2022

ryan99alero commented Nov 13, 2022

longzheng commented Nov 13, 2022

ryan99alero commented Nov 14, 2022

longzheng commented Nov 14, 2022

ryan99alero commented Nov 14, 2022

ryan99alero commented Nov 14, 2022

longzheng commented Nov 14, 2022

ryan99alero commented Nov 14, 2022

nanosonde commented Nov 14, 2022 •

edited

ryan99alero commented Nov 14, 2022

longzheng commented Nov 15, 2022

mcm246 commented Feb 2, 2023

longzheng commented Feb 2, 2023 •

edited

Sunoo commented Feb 2, 2023

mcm246 commented Feb 2, 2023

longzheng commented Feb 2, 2023

nanosonde commented Feb 24, 2023

350d commented Aug 1, 2023

Add support for two-way audio for SIP based video doorbell #1355

Are you sure you want to change the base?

Add support for two-way audio for SIP based video doorbell #1355

Conversation

longzheng commented Nov 8, 2022

ryan99alero commented Nov 12, 2022

mrMiimo commented Nov 12, 2022 • edited

longzheng commented Nov 12, 2022

ryan99alero commented Nov 13, 2022

ryan99alero commented Nov 13, 2022 • edited

longzheng commented Nov 13, 2022

ryan99alero commented Nov 13, 2022

longzheng commented Nov 13, 2022

ryan99alero commented Nov 14, 2022

longzheng commented Nov 14, 2022

ryan99alero commented Nov 14, 2022

ryan99alero commented Nov 14, 2022

longzheng commented Nov 14, 2022

ryan99alero commented Nov 14, 2022

nanosonde commented Nov 14, 2022 • edited

ryan99alero commented Nov 14, 2022

longzheng commented Nov 15, 2022

mcm246 commented Feb 2, 2023

longzheng commented Feb 2, 2023 • edited

Sunoo commented Feb 2, 2023

mcm246 commented Feb 2, 2023

longzheng commented Feb 2, 2023

nanosonde commented Feb 24, 2023

350d commented Aug 1, 2023

mrMiimo commented Nov 12, 2022 •

edited

ryan99alero commented Nov 13, 2022 •

edited

nanosonde commented Nov 14, 2022 •

edited

longzheng commented Feb 2, 2023 •

edited