Feature Request: Add multiple events detections #304

AndreiArdelean1 · 2023-03-30T07:40:51Z

Is your feature request related to a problem? Please describe.
I want to be able to detect multiple event types.
I'm not interested in actually highlighting/retrieving the rectangle where this was detected on the camera, just the detection event.

Describe the solution you'd like
I want to be able to detect and distinguish between:

Motion
AI: Person, Pet, Vehicle, Baby Crying, Glass Break, Dog Barks, Cat Meows
Line-Crossing
Camera Tampering

These notifications are sent to the native app.

Describe alternatives you've considered

Create a different sensor for each event type. This has the benefit of also tracking (either now, or maybe in future updates of the camera) the duration that the event was on.
Add attributes to the motion detection sensor binary_sensor that indicate the type of the event

A clear and concise description of any alternative solutions or features you've considered.

Additional context
These features are available on 3MP+ cameras for free, and on 1080p cameras using the paid subscription

Here is the response from GetEventProperties of the ONVIF endpoint on Tapo C225 camera:

<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://www.w3.org/2003/05/soap-envelope" xmlns:SOAP-ENC="http://www.w3.org/2003/05/soap-encoding" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:wsdd="http://schemas.xmlsoap.org/ws/2005/04/discovery" xmlns:chan="http://schemas.microsoft.com/ws/2005/02/duplex" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" xmlns:wsa5="http://www.w3.org/2005/08/addressing" xmlns:xmime="http://tempuri.org/xmime.xsd" xmlns:xop="http://www.w3.org/2004/08/xop/include" xmlns:wsrfbf="http://docs.oasis-open.org/wsrf/bf-2" xmlns:wstop="http://docs.oasis-open.org/wsn/t-1" xmlns:wsrfr="http://docs.oasis-open.org/wsrf/r-2" xmlns:wsnt="http://docs.oasis-open.org/wsn/b-2" xmlns:tt="http://www.onvif.org/ver10/schema" xmlns:ter="http://www.onvif.org/ver10/error" xmlns:tns1="http://www.onvif.org/ver10/topics" xmlns:tds="http://www.onvif.org/ver10/device/wsdl" xmlns:tmd="http://www.onvif.org/ver10/deviceIO/wsdl" xmlns:trt="http://www.onvif.org/ver10/media/wsdl" xmlns:tev="http://www.onvif.org/ver10/events/wsdl" xmlns:tdn="http://www.onvif.org/ver10/network/wsdl" xmlns:timg="http://www.onvif.org/ver20/imaging/wsdl" xmlns:trp="http://www.onvif.org/ver10/replay/wsdl" xmlns:tan="http://www.onvif.org/ver20/analytics/wsdl" xmlns:tptz="http://www.onvif.org/ver20/ptz/wsdl" xmlns:hikwsd="http://www.onvifext.com/onvif/ext/ver10/wsdl" xmlns:hikxsd="http://www.onvifext.com/onvif/ext/ver10/schema">
    <SOAP-ENV:Header>
        <wsa5:Action SOAP-ENV:mustUnderstand="true">http://www.onvif.org/ver10/events/wsdl/EventPortType/GetEventPropertiesResponse</wsa5:Action>
    </SOAP-ENV:Header>
    <SOAP-ENV:Body>
        <tev:GetEventPropertiesResponse>
            <tev:TopicNamespaceLocation>http://www.onvif.org/onvif/ver10/topics/topicns.xml</tev:TopicNamespaceLocation>
            <wsnt:FixedTopicSet>true</wsnt:FixedTopicSet>
            <wstop:TopicSet>
                <tns1:RuleEngine>
                    <CellMotionDetector>
                        <Motion wstop:topic="true">
                            <tt:MessageDescription IsProperty="true">
                                <tt:Source>
                                    <tt:SimpleItemDescription Type="tt:ReferenceToken" Name="VideoSourceConfigurationToken"></tt:SimpleItemDescription>
                                    <tt:SimpleItemDescription Type="tt:ReferenceToken" Name="VideoAnalyticsConfigurationToken"></tt:SimpleItemDescription>
                                    <tt:SimpleItemDescription Type="xsd:string" Name="Rule"></tt:SimpleItemDescription>
                                </tt:Source>
                                <tt:Data>
                                    <tt:SimpleItemDescription Type="xsd:boolean" Name="IsMotion"></tt:SimpleItemDescription>
                                </tt:Data>
                            </tt:MessageDescription>
                        </Motion>
                    </CellMotionDetector>
                    <TamperDetector>
                        <Tamper wstop:topic="true">
                            <tt:MessageDescription IsProperty="true">
                                <tt:Source>
                                    <tt:SimpleItemDescription Type="tt:ReferenceToken" Name="VideoSourceConfigurationToken"></tt:SimpleItemDescription>
                                    <tt:SimpleItemDescription Type="tt:ReferenceToken" Name="VideoAnalyticsConfigurationToken"></tt:SimpleItemDescription>
                                    <tt:SimpleItemDescription Type="xsd:string" Name="Rule"></tt:SimpleItemDescription>
                                </tt:Source>
                                <tt:Data>
                                    <tt:SimpleItemDescription Type="xsd:boolean" Name="IsTamper"></tt:SimpleItemDescription>
                                </tt:Data>
                            </tt:MessageDescription>
                        </Tamper>
                    </TamperDetector>
                    <LineCrossDetector>
                        <LineCross wstop:topic="true">
                            <tt:MessageDescription IsProperty="true">
                                <tt:Source>
                                    <tt:SimpleItemDescription Type="tt:ReferenceToken" Name="VideoSourceConfigurationToken"></tt:SimpleItemDescription>
                                    <tt:SimpleItemDescription Type="tt:ReferenceToken" Name="VideoAnalyticsConfigurationToken"></tt:SimpleItemDescription>
                                    <tt:SimpleItemDescription Type="xsd:string" Name="Rule"></tt:SimpleItemDescription>
                                </tt:Source>
                                <tt:Data>
                                    <tt:SimpleItemDescription Type="xsd:boolean" Name="IsLineCross"></tt:SimpleItemDescription>
                                </tt:Data>
                            </tt:MessageDescription>
                        </LineCross>
                    </LineCrossDetector>
                    <PeopleDetector>
                        <People wstop:topic="true">
                            <tt:MessageDescription IsProperty="true">
                                <tt:Source>
                                    <tt:SimpleItemDescription Type="tt:ReferenceToken" Name="VideoSourceConfigurationToken"></tt:SimpleItemDescription>
                                    <tt:SimpleItemDescription Type="tt:ReferenceToken" Name="VideoAnalyticsConfigurationToken"></tt:SimpleItemDescription>
                                    <tt:SimpleItemDescription Type="xsd:string" Name="Rule"></tt:SimpleItemDescription>
                                </tt:Source>
                                <tt:Data>
                                    <tt:SimpleItemDescription Type="xsd:boolean" Name="IsPeople"></tt:SimpleItemDescription>
                                </tt:Data>
                            </tt:MessageDescription>
                        </People>
                    </PeopleDetector>
                    <TPSmartEventDetector>
                        <TPSmartEvent wstop:topic="true">
                            <tt:MessageDescription IsProperty="true">
                                <tt:Source>
                                    <tt:SimpleItemDescription Type="tt:ReferenceToken" Name="VideoSourceConfigurationToken"></tt:SimpleItemDescription>
                                    <tt:SimpleItemDescription Type="tt:ReferenceToken" Name="VideoAnalyticsConfigurationToken"></tt:SimpleItemDescription>
                                    <tt:SimpleItemDescription Type="xsd:string" Name="Rule"></tt:SimpleItemDescription>
                                </tt:Source>
                                <tt:Data>
                                    <tt:SimpleItemDescription Type="xsd:boolean" Name="IsTPSmartEvent"></tt:SimpleItemDescription>
                                </tt:Data>
                            </tt:MessageDescription>
                        </TPSmartEvent>
                    </TPSmartEventDetector>
                </tns1:RuleEngine>
            </wstop:TopicSet>
            <wsnt:TopicExpressionDialect>http://www.onvif.org/ver10/tev/topicExpression/ConcreteSet</wsnt:TopicExpressionDialect>
            <wsnt:TopicExpressionDialect>http://docs.oasis-open.org/wsn/t-1/TopicExpression/Concrete</wsnt:TopicExpressionDialect>
            <tev:MessageContentFilterDialect>http://www.onvif.org/ver10/tev/messageContentFilter/ItemFilter</tev:MessageContentFilterDialect>
            <tev:MessageContentSchemaLocation>http://www.onvif.org/onvif/ver10/schema/onvif.xsd</tev:MessageContentSchemaLocation>
        </tev:GetEventPropertiesResponse>
    </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Currently, motion detection doesn't work on Tapo C225 #303, but this feature would be nice to have once it is working.

The text was updated successfully, but these errors were encountered:

JurajNyiri · 2023-03-31T13:42:59Z

This integration reuses onvif from Home Assistant. Adding detectors for these would require to expand on it or handlers to be added into https://github.com/home-assistant/core/blob/dev/homeassistant/components/onvif/parsers.py .

JurajNyiri · 2023-03-31T14:03:33Z

Install https://github.com/JurajNyiri/HomeAssistant-Tapo-Control/tree/expand_onvif
You should see a lot more warning messages now on onvif events trigger
Unsupported entities should come up as "No registered handler for event from" or "Unable to parse event from"

Please report them here.

I tested on C200 with my Person Detection enabled and I have not received any events...

NEVdataDyne · 2023-03-31T15:47:56Z

I would like to help as I need my tapo cameras to trigger automations in HA when a person is detected. Until now the only way I could achieve that is to use Alexa between the cameras and HA. With this solution the cameras need to access internet which is not ideal. Using the video stream to detect people on another system is also not a solution because it uses the WiFi so much that it interferes with my Philips Hue remotes.

Could you please tell me how to install https://github.com/JurajNyiri/HomeAssistant-Tapo-Control/tree/expand_onvif ?

JurajNyiri · 2023-03-31T17:31:24Z

NEVdataDyne · 2023-03-31T20:19:24Z

I downloaded the zip
I updated the Tapo custom component to 5.0
Then I replaced all the files custom_components\tapo_control by the ones from the zip using samba share and restarted HA
I didn't see any new entities so I installed the onvif integration and connected it to one of my C225 camera
I didn't see any new entities either (besides the onvif related entities that I was expecting)
In the logs there doesn't seem to be a message like "No registered handler for event from" or "Unable to parse event from"

I am not sure what I should do?

JurajNyiri · 2023-03-31T21:57:32Z

The code in the branch above doesnt have any new entities. However, it will catch any previously uncought events with an error message. If you are not seeing any other events then it looks like the camera is not issueing those events. I will do some more testing later with the subscription.

AndreiArdelean1 · 2023-04-01T11:56:17Z

@NEVdataDyne It looks like you are having the issue #303 (the last error in your comment). Is your motion sensor (classic one) working/triggered in Homeassistant?

@JurajNyiri I'm not seeing any of the errors you mentioned. Could it be that the integration needs to subscribe explicitly for certain events? Something like "on event PERSON_DETECTED post to WEBHOOK".

Also, because of #303 I'm having issues at the moment. I'm changing the file manually inside onvif-zeep-async but it sometimes gets overwritten or some other issue, but the motion detection (classic one) is not very stable for me. This could be the cause I'm not getting any of the new errors.

NEVdataDyne · 2023-04-01T15:32:07Z

@AndreiArdelean1 No motion sensor doesn't seem to work. It doesn't seem to work even with my C110 camera so I am not really sure if I am doing somethng wrong. (I have C225, C320WS and one C110 camera)

AndreiArdelean1 · 2023-04-02T11:30:20Z

I've gotten the integration to work by modifying tapo_control to use a local version of onvif-zeep-async and only motion events are triggered. I've also verified this using Postman and observing the sent events.

Also, the ONVIF integration declares CellMotionDetector, TamperDetector, LineCrossDetector, PeopleDetector, TPSmartEventDetector. All of this should be just boolean values. But, I've only been able to trigger only CellMotionDetector. Since the camera supports other types of detection events/notifications (like pet detection, glass break, vehicle, etc., within the official app) + boundary boxes of the detection inside the video feed, (apparently) are not exposed through ONVIF, I'm guessing the camera uses some Tapo specific API for this. The TamperDetector, LineCrossDetector, and PeopleDetector are most likely discontinued from the ONVIF API or something else is going on.

I'll investigate this further and post updates if I find something else.

AndreiArdelean1 · 2023-04-03T08:33:10Z

I didn't manage to find a way of getting the special detection events, but the following will help with integrating the configuration of these events (enabling and sensitivity) into HA and pytapo.

Here is the request the app makes on start:
{"method":"multipleRequest","params":{"requests":[{"method":"getDetectionConfig","params":{"motion_detection":{"name":"motion_det"}}},{"method":"getDetectionRegion","params":{"motion_detection":{"table":["region_info"]}}},{"method":"getBCDConfig","params":{"sound_detection":{"name":["bcd"]}}},{"method":"getPersonDetectionConfig","params":{"people_detection":{"name":["detection"]}}},{"method":"getVehicleDetectionConfig","params":{"vehicle_detection":{"name":["detection"]}}},{"method":"getPetDetectionConfig","params":{"pet_detection":{"name":["detection"]}}},{"method":"getBarkDetectionConfig","params":{"bark_detection":{"name":["detection"]}}},{"method":"getMeowDetectionConfig","params":{"meow_detection":{"name":["detection"]}}},{"method":"getGlassDetectionConfig","params":{"glass_detection":{"name":["detection"]}}},{"method":"getTargetTrackConfig","params":{"target_track":{"name":["target_track_info"]}}},{"method":"getAlertConfig","params":{"msg_alarm":{"name":["chn1_msg_alarm_info","capability"]}}},{"method":"getAlertPlan","params":{"msg_alarm_plan":{"name":"chn1_msg_alarm_plan"}}},{"method":"getAlertTypeList","params":{"msg_alarm":{"name":"alert_type"}}},{"method":"getAlertEventType","params":{"msg_alarm":{"table":["msg_alarm_type"]}}},{"method":"getAlertConfig","params":{"msg_alarm":{"table":["usr_def_audio"]}}},{"method":"getMsgPushConfig","params":{"msg_push":{"name":"chn1_msg_push_info"}}},{"method":"getMsgPushPlan","params":{"msg_push_plan":{"name":"chn1_msg_push_plan"}}},{"method":"getTamperDetectionConfig","params":{"tamper_detection":{"name":"tamper_det"}}},{"method":"getLinecrossingDetectionConfig","params":{"linecrossing_detection":{"name":["detection","arming_schedule"]}}},{"method":"getLinecrossingDetectionRegion","params":{"linecrossing_detection":{"table":["region_info"]}}},{"method":"getMsgPushEventList","params":{"msg_push":{"table":["msg_push_event"]}}}]}}

A few examples of setting detection configs:

baby cry
{"method":"multipleRequest","params":{"requests":[{"method":"setBCDConfig","params":{"sound_detection":{"bcd":{"enabled":"on","sensitivity":"medium"}}}}]}}
person
{"method":"multipleRequest","params":{"requests":[{"method":"setPersonDetectionConfig","params":{"people_detection":{"detection":{"enabled":"off","sensitivity":"60"}}}}]}}
pet
{"method":"multipleRequest","params":{"requests":[{"method":"setPetDetectionConfig","params":{"pet_detection":{"detection":{"enabled":"off","sensitivity":"60"}}}}]}}
bark
{"method":"multipleRequest","params":{"requests":[{"method":"setBarkDetectionConfig","params":{"bark_detection":{"detection":{"enabled":"on","sensitivity":"50"}}}}]}}
glass break
{"method":"multipleRequest","params":{"requests":[{"method":"setGlassDetectionConfig","params":{"glass_detection":{"detection":{"enabled":"on","sensitivity":"60"}}}}]}}
tamper
{"method":"multipleRequest","params":{"requests":[{"method":"setTamperDetectionConfig","params":{"tamper_detection":{"tamper_det":{"enabled":"on","sensitivity":"medium"}}}}]}}

JurajNyiri · 2023-04-03T08:51:45Z

Updated the code in https://github.com/JurajNyiri/HomeAssistant-Tapo-Control/tree/expand_onvif

You should now see:

Warn message for "async_parse_messages" every time any event is detected
Warn Message Found Event when it found an event it recognizes
Error message "No registered handler for event from" or "Unable to parse event from" when event is found which is not recognized

2023-04-03 08:49:23.920 WARNING (MainThread) [custom_components.tapo_control] async_parse_messages

2023-04-03 08:49:23.921 WARNING (MainThread) [custom_components.tapo_control] Found event ba835a1edb0d0cdf5bcbafab8b732021_tapo_events: {

    'SubscriptionReference': None,

    'Topic': {

        '_value_1': 'tns1:RuleEngine/CellMotionDetector/Motion',

        'Dialect': 'http://www.onvif.org/ver10/tev/topicExpression/ConcreteSet',

        '_attr_1': {

    }

    },

    'ProducerReference': None,

    'Message': {

        '_value_1': {

            'Source': {

                'SimpleItem': [

                    {

                        'Name': 'VideoSourceConfigurationToken',

                        'Value': 'vsconf'

                    },

                    {

                        'Name': 'VideoAnalyticsConfigurationToken',

                        'Value': 'VideoAnalyticsToken'

                    },

                    {

                        'Name': 'Rule',

                        'Value': 'MyMotionDetectorRule'

                    }

                ],

                'ElementItem': [],

                'Extension': None,

                '_attr_1': None

            },

            'Key': None,

            'Data': {

                'SimpleItem': [

                    {

                        'Name': 'IsMotion',

                        'Value': 'true'

                    }

                ],

                'ElementItem': [],

                'Extension': None,

                '_attr_1': None

            },

            'Extension': None,

            'UtcTime': datetime.datetime(1970, 1, 1, 0, 0, tzinfo=<isodate.tzinfo.Utc object at 0x7f663048c580>),

            'PropertyOperation': 'Changed',

            '_attr_1': {

        }

        }

    }

}

AndreiArdelean1 · 2023-04-03T09:48:01Z

I've already done something similar, but the only thing that changed was the Data.SimpleItem.Value (+ UtcTime). No new event was received by tapo_control, even though the notification was sent to the app. Probably the camera doesn't sent any of the new events using ONVIF.

JurajNyiri · 2023-04-03T11:15:47Z

I have added to main branch (not yet released to pypi):

get/set VehicleDetection
get/set Cruise
get/set MeowDetection
get/set BarkDetection
get/set PetDetection
get/set BabyCryDetection
get/set TamperDetection
get/set GlassBreakDetection

I was not able to test:

setBarkDetection
setPetDetection
setGlassBreakDetection
setMeowDetection
setVehicleDetection
getBarkDetection
getPetDetection
getGlassBreakDetection
getMeowDetection
getVehicleDetection
getCruise

@AndreiArdelean1 could you please test above?

Edit: Released as pytapo 3.1.8.

JurajNyiri · 2023-04-03T13:16:51Z

@AndreiArdelean1 now released in https://github.com/JurajNyiri/HomeAssistant-Tapo-Control/releases/tag/5.1.0.beta.1 Please test above, my devices do not support it.

AndreiArdelean1 · 2023-04-03T17:50:00Z

They all work except for getCruise. getCruise always is {'patrol': {'patrol': {'action': 'idle'}}} regardless if it is idle or doing the cruise. The HA integrations also works. 🎉

AndreiArdelean1 · 2023-04-07T07:20:57Z

I've been investigating this issue and here is what I've found so far:

ONVIF:

the ONVIF endpoint declares IsMotion, IsTamper, IsLineCross, IsPeople, IsTPSmartEvent
for IsTamper, IsLineCross I'm not sure if they are triggered (I don't remember if I tested this), but IsPeople is not triggered
Pet, Vehicle, Baby Crying, Glass Break, Dog Barks, Cat Meows, are not even declared in the ONVIF endpoint

UDP:

There aren't any multicast packets sent when an event is triggered
The app has the option to highlight people, pets, etc. in the live stream. The only place where this could take place (from the network packets received by the app) is inside the video/mp2t in the live stream from port 8800 (https://md.depau.eu/s/r1Ys_oWoP#Live-streaming, https://drmnsamoliu.github.io/video.html). There are no additional application/json received besides the initial ones (informing of the privacy mode overlay). I've not been able to decode the sent JSON packets, only the received ones, as they are encrypted. Most likely the highlighted zones are already included in the video.

API:

the App has a feature to view recordings where it also indicates the type of event triggered.
when the app enters that screen, here are the calls made:

{"method":"multipleRequest","params":{"requests":[{"method":"searchDateWithVideo","params":{"playback":{"search_year_utility":{"channel":[0],"end_date":"20230431","start_date":"20230401"}}}}]}}
{"method":"multipleRequest","params":{"requests":[{"method":"getSdCardStatus","params":{"harddisk_manage":{"table":["hd_info"]}}},{"method":"getCircularRecordingConfig","params":{"harddisk_manage":{"name":"harddisk"}}}]}}
{"method":"multipleRequest","params":{"requests":[{"method":"getClockStatus","params":{"system":{"name":"clock_status"}}},{"method":"getDstRule","params":{"system":{"name":"dst"}}},{"method":"getTimezone","params":{"system":{"name":["basic"]}}}]}}
{"method":"multipleRequest","params":{"requests":[{"method":"searchDateWithVideo","params":{"playback":{"search_year_utility":{"channel":[0],"end_date":"20230331","start_date":"20230301"}}}}]}}
{"method":"multipleRequest","params":{"requests":[{"method":"searchVideoWithUTC","params":{"playback":{"search_video_with_utc":{"id":1,"start_index":0,"channel":0,"start_time":1680555600,"end_time":1680641999,"end_index":99}}}}]}}
{"method":"multipleRequest","params":{"requests":[{"method":"searchDetectionList","params":{"playback":{"search_detection_list":{"start_index":0,"channel":0,"start_time":1680555600,"end_time":1680641999,"end_index":99}}}}]}}
{"method":"multipleRequest","params":{"requests":[{"method":"searchDateWithVideo","params":{"playback":{"search_year_utility":{"channel":[0],"end_date":"20230231","start_date":"20230201"}}}}]}}

What can be used, for event detection is the call:

{"method":"searchDetectionList","params":{"playback":{"search_detection_list":{"start_index":0,"channel":0,"start_time":1680555600,"end_time":1680641999,"end_index":99}}}}

and returning:

{'playback': {'snapshot_enable': True, 'search_detection_list': [{'start_time': 1680725544, 'end_time': 1680725662, 'alarm_type': 6}, {'start_time': 1680725693, 'end_time': 1680725802, 'alarm_type': 6}, {'start_time': 1680725826, 'end_time': 1680725856, 'alarm_type': 6}, {'start_time': 1680725917, 'end_time': 1680725947, 'alarm_type': 6}], 'total_num': 4}}

A few things to note here:

the end time extends if multiple events are triggered in quick succession
the alarm type also updates. People, pet detection is preferred over motion events.
the start_time and end_time define a range and the camera returns the events that have at least 1s within that range (event.start_time can be before request.start_time, and the same for end_time)

I've written this python script to get the events from the last 10 min to test it:

secondsFrom1970 = tapo.executeFunction(
            "getClockStatus", {"system":{"name":"clock_status"}}
          )["system"]["clock_status"]["seconds_from_1970"]
print(secondsFrom1970)
searchStart = secondsFrom1970 - (10 * 60) # 10 min
searchEnd = secondsFrom1970 + 10
print(tapo.executeFunction(
  "searchDetectionList", {"playback":{"search_detection_list":{"start_index":0,"channel":0,"start_time":searchStart,"end_time":searchEnd,"end_index":99}}}
))

Maybe this method of detection motion could be used to detect motion on #273.
This could be done by:

making the above call every 0.5-1s and getting the last event (named here lastEvent) from the list and later storing it to savedLastEvent.
if the start_time of lastEvent and savedLastEvent are not the same, a new event was triggered
if the alarm_type of lastEvent and savedLastEvent are not the same, a new event was triggered
if savedLastEvent has completed, a new event was triggered
if secondsFrom1970 is within 30s (this is the default/minimum duration of an event) or 10s from the end_time, the event could still be active, otherwise the lastEvent can be marked as completed

Instead of making a call periodically, the ONVIF motion event could be used to trigger the call to searchDetectionList. BUT, the motion event is not triggered if only an AI event is detected. If the motion sensitivity is set high enough, the 2 events would be triggered in close succession and this would probably work.

Probably the best solution would be a combination of making a call periodically and watching for ONVIF motion events.

Other possible methods not yet discovered/implemented on the camera:

adding the missing events to ONVIF
the camera to emit a multicast broadcast (preferably encrypted) whenever something changes or just on events triggered
a way to register a webhook from the camera API for this
a way to register a webhook from the Tapo server to HA. Something similar to https://ifttt.com/tplink_tapo/details, but from what I could find, the API is only available to select developers.
opening a socket connection to the camera for events (not likely to be done in the future, or to already exist)
intercept the calls made by the camera to Tapo servers for notification (if possible, too complicated for the average user for the purpose of this integration)
register webhooks with IFTTT (tested and working). The only problem here is that the free plan only allows 5 automations.
register webhooks with Alexa

NEVdataDyne · 2023-04-07T13:26:13Z

Maybe in "Other possible methods not yet discovered/implemented" you can add "intercept the calls made by the camera to Alexa servers for notification" (for people detection). For some reason Alex seems to be the only service other than Tapo to get notified when a person is detected (and this is the only event Alexa can get).

AndreiArdelean1 · 2023-04-13T07:53:47Z

I've written this script (attached) to pull events from the camera. It works really well. I'd implement a proof of concept for Tapo-Control, but I'm not very familiar with HomeAssistant integrations. Maybe this method could be used alogside the existing ONVIF events for detecting motion.
Notes: The 2 files need to be under the same directory.
Archive.zip

JurajNyiri · 2023-04-13T10:15:02Z

Thank you @AndreiArdelean1 . I played with the script, it had a bug where it was calculating incorrect start time for cameras in different timezone to UTC-0. I adjusted a couple of things and released new function getEvents to pytapo 3.1.11.

I am testing with following script:

from pytapo import Tapo
import time
from datetime import datetime
import os

# Camera
user = os.environ.get("user")  # admin user
password = os.environ.get("password")  # cloud password
host = os.environ.get("ip")  # ip of the camera, example: 192.168.1.52

tapo = Tapo(host, user, password, password)

epoch_time = int(time.time())
secondsFrom1970 = epoch_time

minSearchStart = secondsFrom1970 - (60 * 60)

searchStart = minSearchStart # can be used as opt parameter in tapo.getEvents()
searchEnd = secondsFrom1970 + 60  # maybe add a few seconds? # can be used as opt parameter in tapo.getEvents()

while True:
    events = tapo.getEvents()
    if events:
        lastEvent = events[-1]

        newLastEventDictionary = {
            'startTime': lastEvent["start_time"],
            'endTime': lastEvent["end_time"],
            'alarmType': lastEvent["alarm_type"],
        }
        print(datetime.now())
        print(lastEvent)
        print(datetime.fromtimestamp(newLastEventDictionary['startTime']))
        print(datetime.fromtimestamp(newLastEventDictionary['endTime']))
        print("")
    else:
        print("none")
    time.sleep(1)

I tested on 2 C200s. One HW V1.0 and another one 3.0. The 3.0 also has tapo cloud subscription.
I found that:

Events are not reported reliably, even if they are visible in the timeline of recordings.
Events are reported with start time in the future

I tested also with your script only to verify I haven't made any mistakes when writing the new function.

Could you try it and let me know if it works with your cameras?

AndreiArdelean1 · 2023-04-13T13:06:41Z

I get something like:

2023-04-13 15:57:14.501225
{'start_time': 1681390629, 'end_time': 1681390659, 'alarm_type': 2, 'startRelative': 4.0, 'endRelative': -26.0}
2023-04-13 15:57:09
2023-04-13 15:57:39

start time is always in the past (startRelative > 0)
for me, the events are received reliably. An event being a change in either start_time, end_time, or alarm_type of the last event

What I've noticed, if multiple sources (script and Tapo app) are requesting the events at once, one of them could return an empty list.

AndreiArdelean1 · 2023-04-14T15:06:53Z

Off-topic, but I'm not sure where I should post this:
All the possible values for alarm_type from searchDetectionList:

allKnownAlarmTypes = {
    1: "Timing",
    2: "Motion",
    3: "TAMPER",
    4: "Line Crossing",
    5: "Area Intrusion", 
    6: "Person",
    7: "Baby Cry",
    8: "Vehicle",
    9: "Pet",
    10: "Ring Alarm",
    11: "Dog Bark",
    12: "Cat Meow",
    13: "Alarm Glass",
    14: "Alarm Smoke",
    15: "Deliver Package",
    16: "Pickup Package",
    17: "Dollbell Ring Missed",
    18: "Dollbell Ring Answered",
    19: "Anti Theft",
}

I'm not sure which work, which don't, or what they mean. I've extracted them from the Android app.
The raw values:

1: TIMING 
2: MOTION 
3: TAMPER 
4: LINECROSS 
5: AREAINTRUSION 
6: HUMAN 
7: BABYCRY 
8: VEHICLE 
9: PET 
10: RINGALARM 
11: ALARMBARK 
12: ALARMMEOW 
13: ALARMGLASS 
14: ALARMSMOKE 
15: DELIVERPACKAGE 
16: PICKUPPACKAGE 
17: DOLLBELLRINGMISSED 
18: DOLLBELLRINGANSWERED 
19: ANTITHEFT

JurajNyiri · 2023-05-04T14:25:06Z

@AndreiArdelean1 could you please test this script with the pytapo 3.1.13 to verify it still works? I think I managed to fix the timezone issue and it seems to be working fairly well now for me.

AndreiArdelean1 · 2023-05-05T06:35:35Z

It works!

fightforlife · 2023-05-07T12:49:46Z

When I try to run the script above on the latest python docker image I get the following error:
File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 517, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='192.168.1.81', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1002)')))

Do you have a tip? Maybe it has something todo with the IP not being a hostname? Is there a way to set the request to insecure?

rmtrane · 2023-06-06T16:55:10Z

@AndreiArdelean1 Just to clarify, you were able to extract the AI detected events (person, pet, vehicle, baby crying, etc.)? I was playing around for a minute with the getEvents function, but couldn't get it to work. Just wanted to make sure it would be worth the time to actually dive in... Thank you!

AndreiArdelean1 · 2023-06-07T16:46:02Z

Hey, yes, it is working. It did work for motion, person, and pet, but it wasn't working for tamper detection (Or I didn't manage to get it in my testing as that type of detection was a bit hard to trigger). Sounds detection (meow, bark, baby, glass break) I couldn't trigger the actual event during testing so I can't say if it is working. But, if an event is shown with the correct icon in the playback list in the official app, then you should also be able to detect it with this function.

You could:

every x seconds request the most recent events. The downside is that it introduces a load on the camera, and since the camera has limited resources (CPU, memory, etc.) the official app may become unresponsive, and it may cause other unknown issues
when an event is detected with ONVIF, wait for 1s (or a smaller amount of time to make sure the event registers) and request the latest event from the camera to get the actual event type.

I've only tested method 1. When investigating I had some issues with ONVIF so I couldn't test method 2.

rmtrane · 2023-06-07T17:06:51Z

Thank you for letting me know! I'm mainly interested in the sounds detection, so might try to get this to work.

(In case you're interested, I have had success in triggering the events resulting in notifications through the Tapo app by simply using youtube videos of baby crying, dog barking, etc.)

@JurajNyiri Any plans to implement something like this in the Home Assistant integration? I will probably try to do something ad-hoc, but would be really cool to have this in the integration 😊

JurajNyiri · 2023-08-19T14:19:21Z

I was looking into this again in hope to implement it and tested it on C200 HW 1.0 1.3.6 Build 230424 Rel.77225n(4555) and C200 HW 3.0 1.3.6 Build 230424 Rel.77225n(4555) with this script and I am only getting the event once it stops - not when it starts. Meaning it shows up with endRelative 0 but never with missing end or start being around 0. Is this different for you @AndreiArdelean1 ?

AndreiArdelean1 · 2023-08-21T05:58:36Z

For me (C225) it detects events as soon as they are detected and sets the end date to a date in the future
I'd recommend not implementing this script directly into HA as it strains the cameras already limited processing power and the whole camera may freeze or even crash. This script also doesn't always work if you try to perform the call to get the events from the official app. This is probably because the call takes some time to be processed on the camera, and if another request is received by the camera during that time, the second request would fail.

dvpereira · 2023-08-21T17:17:29Z

@JurajNyiri I ran the script and noticed that it only returns the event about 2s AFTER the detection window ends. I tested it on my TAPO C200. I'm going to test on the C310 in a little while I'll return here.

2023-08-21 14:11:50.756539
{'start_time': 1692637890, 'end_time': 1692637908, 'alarm_type': 2, 'startRelative': 20, 'endRelative': 2}
2023-08-21 14:11:30
2023-08-21 14:11:48

dvpereira · 2023-08-21T17:22:21Z

My C310 answer:

2023-08-21 14:21:02.550051
{'start_time': 1692638422, 'end_time': 1692638462, 'alarm_type': 2, 'startRelative': 40, 'endRelative': 0}
2023-08-21 14:20:22
2023-08-21 14:21:02

I'm doing some research to try to figure out how (if) can we change this behavior...

erasmus83 · 2023-08-30T16:31:02Z

Also very interested in this feature - I have C210 cameras and always happy to help test

Myrddraal · 2023-11-20T21:55:50Z

For those looking to use the built-in Tapo person detection with Home Assistant, until we can do so through this integration, I am using a workaround:

I have a Tapo TC65 and a Tapo smart socket P100. In the Tapo app, I can set the smart socket to switch on based on AI person detection.
In Home Assistant, I then use the state of the smart socket as a proxy for person detection. It switches itself back off, so that the next person detection event can switch it on again.

Obviously not an ideal workaround, but it works well enough for me for now!

NEVdataDyne · 2023-11-20T22:38:22Z

That's interesting, I am currently using the person detection routine in Alexa to turn a Phillips hue light on that gets detected by HA. The most cumbersome way of detecting a person i suppose but didn't find a better way. Your method would allow me a least to ditch Alexa who tries to sell me batteries for my hue remotes when they run out...

NEVdataDyne · 2023-11-21T06:52:53Z

@Myrddraal can you check if your solution works without internet? I know the camera's person detection is done off line but I wonder if the internet is necessary for your tapo automation to work

Myrddraal · 2023-11-21T09:15:44Z

@NEVdataDyne I haven't tried, but I'm fairly sure that it does rely on an internet connection. As I understand it, you can view the camera feed on the same network, without internet, but controlling other devices requires an internet connection.

cncb-gh · 2024-01-22T19:51:32Z

Does it seem unlikely that we will ever be able to get Person Detection events? This is the main issue preventing me from going all local.

JurajNyiri · 2024-01-22T19:54:45Z

The best would be to ask tplink to add it to the onvif standard. Then it will work across all the 3rd party solutions and applications.

cncb-gh · 2024-01-23T22:04:54Z

TP-Link support said they passed on the request to R&D but you know how that goes...

akomelj · 2024-02-12T16:52:07Z

Just a small side note based on my observations:

I have a Tapo C220 cam. The cam supports two types of notification subscriptions:

pull point subscriptions where ONVIF client polls the cam-provided URL for events;
base notification subscriptions where the ONVIF cam connects to the client-provided URL and posts events to the client.

I noticed, in accordance with all the above observations, that the cam does not publish events on any topic apart from Motion when pull point event polling is used.

However, when the second method is used, the cam sends additional events. I can get notifications when AI detects a person in motion (EVENT: 2024-02-11T13:07:05.000Z RuleEngine/CellMotionDetector/People PROP:Changed SRC:VideoSourceConfigurationToken, VideoAnalyticsConfigurationToken, Rule=vsconf, VideoAnalyticsToken, MyPeopleDetectorRule DATA:IsPeople=true). 👍

I haven't tested for other event types yet (pets, cars, noise) as this is a bit harder to do, but I will update this comment if I observe any other event types on the listener.

EDITED 2024-02-15: The cam unfortunately does not publish pet-related events to web hook - i.e. cam detects a cat and shows overlay in the app but does not publish TPSmartEvent via ONVIF notification system.

rct · 2024-03-13T22:51:16Z

I just got this custom integration set up for a C120 and am I trying to digest the current state of this issue.

1st question, I think the C120 is fairly new hardware introduced in 2023 based on reviews. Is there any testing I should be doing against it?

EDIT

Two more questions:

Are there feature requests in the TP-Link community forum that it would help to have people upvote/chime in on?
Are their Home Assistant ONVIF feature or pull requests worth tracking?

AndreiArdelean1 assigned JurajNyiri Mar 30, 2023

This comment was marked as off-topic.

Sign in to view

fightforlife mentioned this issue May 5, 2023

ONVIF Motion Events / Binary sensor not appearing on recent firmwares #312

Closed

This was referenced Aug 18, 2023

Feature Request: Smart Detection Entities #386

Closed

Feature Request: Support Floodlight Control on Tapo C320WS HW2.0 #377

Closed

JurajNyiri added Enhancement New feature or request Help wanted Extra attention is needed labels Aug 20, 2023

JurajNyiri mentioned this issue Aug 22, 2023

Human Detection Notification JurajNyiri/pytapo#72

Closed

JurajNyiri mentioned this issue Sep 17, 2023

Feature Request: Baby Crying binary sensor #407

Closed

This was referenced Oct 9, 2023

Feature Request: Add Sensor for person detection #416

Closed

Feature Request: Additional binary sensors #424

Closed

Feature Request: #425

Closed

nafets2 mentioned this issue Oct 29, 2023

Feature Request: Motion Dectection for new Cameras with AI-Feature (Persons, Cars, Pets) eg. C225, C325WS #434

Closed

JurajNyiri mentioned this issue Dec 28, 2023

Feature Request: HDR Switch #480

Open

JurajNyiri mentioned this issue Mar 14, 2024

Extra methods for H200 and C420S2 battery cameras JurajNyiri/pytapo#105

Closed

Feature Request: Add multiple events detections #304

Feature Request: Add multiple events detections #304

Comments

AndreiArdelean1 commented Mar 30, 2023 • edited

JurajNyiri commented Mar 31, 2023

JurajNyiri commented Mar 31, 2023 • edited

NEVdataDyne commented Mar 31, 2023 • edited

JurajNyiri commented Mar 31, 2023

NEVdataDyne commented Mar 31, 2023 • edited

JurajNyiri commented Mar 31, 2023 • edited

This comment was marked as off-topic.

AndreiArdelean1 commented Apr 1, 2023 • edited

NEVdataDyne commented Apr 1, 2023

AndreiArdelean1 commented Apr 2, 2023

AndreiArdelean1 commented Apr 3, 2023 • edited

JurajNyiri commented Apr 3, 2023

AndreiArdelean1 commented Apr 3, 2023

JurajNyiri commented Apr 3, 2023 • edited

JurajNyiri commented Apr 3, 2023

AndreiArdelean1 commented Apr 3, 2023

AndreiArdelean1 commented Apr 7, 2023 • edited

NEVdataDyne commented Apr 7, 2023

AndreiArdelean1 commented Apr 13, 2023

JurajNyiri commented Apr 13, 2023 • edited

AndreiArdelean1 commented Apr 13, 2023 • edited

AndreiArdelean1 commented Apr 14, 2023

JurajNyiri commented May 4, 2023

AndreiArdelean1 commented May 5, 2023

fightforlife commented May 7, 2023 • edited

rmtrane commented Jun 6, 2023 • edited

AndreiArdelean1 commented Jun 7, 2023 • edited

rmtrane commented Jun 7, 2023

JurajNyiri commented Aug 19, 2023 • edited

AndreiArdelean1 commented Aug 21, 2023

dvpereira commented Aug 21, 2023

dvpereira commented Aug 21, 2023

erasmus83 commented Aug 30, 2023

Myrddraal commented Nov 20, 2023

NEVdataDyne commented Nov 20, 2023

NEVdataDyne commented Nov 21, 2023

Myrddraal commented Nov 21, 2023

cncb-gh commented Jan 22, 2024

JurajNyiri commented Jan 22, 2024

cncb-gh commented Jan 23, 2024 • edited

akomelj commented Feb 12, 2024 • edited

rct commented Mar 13, 2024 • edited

AndreiArdelean1 commented Mar 30, 2023 •

edited

JurajNyiri commented Mar 31, 2023 •

edited

NEVdataDyne commented Mar 31, 2023 •

edited

NEVdataDyne commented Mar 31, 2023 •

edited

JurajNyiri commented Mar 31, 2023 •

edited

AndreiArdelean1 commented Apr 1, 2023 •

edited

AndreiArdelean1 commented Apr 3, 2023 •

edited

JurajNyiri commented Apr 3, 2023 •

edited

AndreiArdelean1 commented Apr 7, 2023 •

edited

JurajNyiri commented Apr 13, 2023 •

edited

AndreiArdelean1 commented Apr 13, 2023 •

edited

fightforlife commented May 7, 2023 •

edited

rmtrane commented Jun 6, 2023 •

edited

AndreiArdelean1 commented Jun 7, 2023 •

edited

JurajNyiri commented Aug 19, 2023 •

edited

cncb-gh commented Jan 23, 2024 •

edited

akomelj commented Feb 12, 2024 •

edited

rct commented Mar 13, 2024 •

edited