ISAPI 'Invalid Operation' Response

[ajhodges]

This is almost certainly an issue with my device and not pyHik, but I thought I'd open a thread about it.

I'm playing around with possibly adding support for the Hikvision DS-KB6003-WIP Doorbell cam to pyHik, and hopefully catching the doorbell event. However, I've noticed some stability issues with the alertStream on my device.

Sometimes the /ISAPI/Event/notification/alertStream endpoint returns a healthy response that looks like this:

<EventNotificationAlert version="1.0" xmlns="urn:psialliance-org">
    <ipAddress>192.168.1.100</ipAddress>
    <portNo>80</portNo>
    <protocol>HTTP</protocol>
    <macAddress>*****************</macAddress>
    <dateTime>2018-01-31T07:21:21-05:00</dateTime>
    <activePostCount>1</activePostCount>
    <eventType>other</eventType>
    <eventDescription>Heartbeat</eventDescription>
    <eventState>inactive</eventState>
    <Extensions version="1.0" xmlns="urn:psialliance-org">
        <serialNumber>****************************************</serialNumber>
    </Extensions>
</EventNotificationAlert>

but then occasionally it will only send this response in a HTTP 403 error (which of course pyHik doesn't like):

<?xml version="1.0" encoding="UTF-8" ?>
<ResponseStatus version="1.0" xmlns="urn:psialliance-org">
    <requestURL>/ISAPI/Event/notification/alertStream</requestURL>
    <statusCode>4</statusCode>
    <statusString>Invalid Operation</statusString>
    <subStatusCode>invalidOperation</subStatusCode>
</ResponseStatus>

Once it starts sending invalidOperation responses, it will only return those responses for some undefined amount of time until it eventually starts working again (I haven't timed it). It seems completely random. I have saved the HTTP request in Postman (so not even using Python/pyHik at this point) and can verify that I observe the same behavior there as well, so I think this is a server (device) issue.

The API documentation isn't too helpful about this message. I don't think it's an authentication issue since I'm sending the same basic auth credentials with every request, and it's a 403 (forbidden) not a 401 (unauthorized).

I'm just mostly wondering if @mezz64 you have seen anything like this in your testing with your devices. I am thinking about trying to reach out to Hikvision to see if I can get an explanation... barring any magic solution I would probably have to add some sort of retry logic to handle this in order to add support for this device, unless my device is defective or this is fixed in a future firmware update.

[mezz64]

Can I assume your camera is running the v5.5.0 series of firmware? I'm wondering if the 403 responses showing up are what are causing the issues for others with that same firmware in the other open issue I currently have. I never seen that response on the older versions.

Can you verify that if you disconnect and reconnect once the 403 responses start showing up you return to a healthy stream? If so I can add some detection for that response and trigger a reconnect to keep things moving.

[ajhodges]

I believe the firmware version I am on is (latest currently available) V1.4.62 build 171219. I can double check on that.

So, I'm not sure how your devices behave (and this has crossed my mind as a potential cause of this issue), but when I hit /ISAPI/Event/notification/alertStream it always returns almost immediately, certainly within 500ms. Is this supposed to be an actual streaming endpoint that stays open indefinitely? Because I have not gotten it to behave that way yet. So maybe it's possible that the firmware on the doorbell doesn't support streaming requests (which would make the Invalid Operation response make more sense). But then again, I don't think there is really anything special about a 'streaming' HTTP request, is there? I think it's mostly a server-side thing..

But to answer your question, no, I still get the 403 response even on subsequent HTTP requests. I will have some more time to play around with it tonight.

[mezz64]

You firmware versioning points to a different series for these devices. I'd just assumed they used the same thing they use on all their other cameras, but that may not be the case.

If I curl my alertStream url I receive a continuous stream of events. The connection will stay open until either I close it or a network error occurs interrupting the connection. This syncs up with the description of the endpoint in the only published API documentation i'm aware of:

This function is used to get an event notification alert stream from the media device via HTTP or HTTPS. This function does not require that a client/VMS system be added as an HTTP(S) destination on the media device. Instead, the client/VMS system can call this API to initialize a stream of event information from the device. In other words, a connection is established with the device when this function is called, and stays opento constantly receive event notifications.

If yours is constantly disconnecting it's going to make it nearly impossible to render a reliable stream though. Since you're testing things out with postman, have you tried using digest auth, assuming your firmware version supports it?

[ajhodges]

Here are the two firmwares I've tried so far, they're two firmwares from two different resellers and seem to be the latest builds available:

Same behavior with both. I also tried using digest auth, no change. I've noticed that the other /ISAPI/Event endpoints seem to be unimplemented, many of them return 'Can't locate the url'. I am guessing that this functionality is just unfortunately not supported in this firmware :(

It's a shame, because this doorbell coupled with HASS could be a total Ring-killer. I will keep an eye out for new firmware and keep my fingers crossed for this functionality.

[mezz64]

Another route you could take (assuming the app notifies you quickly when someone triggers an event) is to run the app requests through a proxy server (charles or similar) and see if you can capture the endpoint the app is connecting to or the method it's using.

Only tricky part is that if there are any SSL calls involved you'll need to use a pre-Nougat phone for android so you can trust the Charles SSL cert for things to work. Not sure if iphones are any different.

[randellhodges]

I have this camera and after @ajhodges notified me of this issue, I did some testing.

curl -u user:pass -w "\n%{time_connect}\n%{time_starttransfer}\n%{time_total}\n" -N http://[redacted]/ISAPI/Event/notification/alertStream

Yields a 2 second time_total and a open/close connection, aka, it does not actually stream. I am using the v1.4.62 build 171219 firmware.

I have an NVR. I wonder if pointing to it and letting it handle things would allow this camera to work in HA.

[ajhodges]

Just an update to crush any dreams that might have been formed by the changelog of the latest update for this DS-KB6003-WIP doorbell cam. The changelog lists "Support ONVIF and ISAPI protocols" under new features. I updated my doorbell to this latest firmware (V1.4.62 build 180515) and still am getting the 403 error. Sorry!