Support for car and person detection

[ldwedari]

Hi.
Thanks for this great component.
I would recommend to add some comment to the unsupported cameras section in the Readme.md explaining why they are unsupported. Basically because the only work with Reolink NVRs and don't have ONVIF. Not giving an explanation is quite confusing for owners of those cameras.
Also I'd like to ask if this component supports the latest Reolink smart cameras that can detect persons and cars. Being able to create sensor for this type of events in Home Assistant would be amazing.
Finally, what is the difference between reolink and reolink_dev?
(BTW, a discord channel would be nice)

[fwestenberg]

I will add some info as soon as NVR has good support. Thanks for that.
For the person and car detection I will start working on that soon, this needs some research first.

Reolink_dev is the HACS repository. Reolink is the Python library.

[spripe]

Hi,
What about car/person detection for cameras that supports this without NVR. Like RLC-810A cameras?

[fwestenberg]

For car/person detection I need some time. Please be patient.

[theBOFH]

+1 on this, having the detection in HomeAssisatant would be awesome for sure.
@fwestenberg thanks for all the time and effort put into developing the integration, really appreciated.

Currently researching camera systems and had been looking at the Hikvision cameras with the person detection but reolink looks to be way more affordable.

[tomlut]

Jaysus dood! I nearly had a heart attack when I saw an email notification from theBOFH.

[ScottRoach]

Does anyone have any idea if this information is exposed anywhere? I only see motion events (not person/car specific) in the onvif messages. I've looked at the JSON in the web view and nothing stands out.

Looking through the desktop app's obfuscated code I have a feeling it is only exposed through the extended attributes in the Baichuan protocol instead of metadata/onvif/rtsp.

[fwestenberg]

Hi Scott, thanks for checking this out! I have traces the data traffic with Wireshark, and also see this. I'm afraid the cgi api doesn't offer this info.

[ldwedari]

I contacted Reolink support with this questions:

Will Reolink provide ONVIF Profile C events about person and car detection? And when?
This ONVIF support is important because RLC-810A could be used with non cloud based smart home software like OpenHab, Home Assistant and Domoticz. It could even be integrated in other professional systems.

And they answer was:

1. You could use the smart detection function for RLC-810A via Reolink App/Client only.

If you integrate the camera to Alexa/Google home, you could live view the camera on the Alexa/Google home device only. You couldn't use the smart detection function.
It is the same as the ONVIF protocol.

2. Sorry that we don't know the available time for it. But we will forward your request to our engineers.
   You may subscribe our emails to get the news.

So as it is now, I would not recommend the camera for anyone wanting to integrate it in Home Assistant.

[tomlut]

I wouldn't go that far. It's still a very good camera for the price.

Do their NVRs even work with person and car detection?

[fwestenberg]

The engineers at Reolink currently work an the API so we can use person/vehicle detection within HA. The support they provide is very great, and you just have to be a little more patient. I agree the camera's they provide are very good for the price. And beside: look what is already supported now, compared to a lot of other camera's!

[ldwedari]

I didn't know they were already working in that API. It did not sound line that from their response.
Btw, I also think they are good/value for money. It just that the only way to implement AI detection in HA is to use DOODS. I hope this changes soon.

[badabing2005]

I agree with @fwestenberg 100%, Reolink support alone is worth the price, I never had an issues with their support, they reply same day / next day, even on weekends and Holidays, and they do actually help, they take every customer issue seriously.
That coupled with good camera and very good pricing makes it a no brainer, and then we have @fwestenberg adding integration like this and we have a recipe for success.
Just be patient, I was told by their support that their engineers are working on firmware update to support onvif better.

[Tomkee]

@fwestenberg thank you for your work. I am waiting for the official Reolink HA component.

[jrtraynor]

Hi,
I too would like to see the ai detection to be available in HA. I have the 520A camera, don't know if this is helpful or you're already aware. For me I can go to following url instead of the GetMdState one:
http://192.168.86.2/api.cgi?cmd=GetAiState&user=X&password=X
which returns
[
{
cmd: "GetAiState",
code: 0,
value: {
channel: 0,
face: {
alarm_state: 0,
support: 0
},
people: {
alarm_state: 1,
support: 1
},
vehicle: {
alarm_state: 0,
support: 1
}
}
}
]
where the alarm_state becomes 1 on detection. ( here someone walked by so "people" was detected.) then goes back to 0. Same as the GetMdState json

hth

James

[ldwedari]

Great! Then since with ONVIF we can get a normal motion notification it should be possible to get the type of alarm by calling that GetAiState endpoint right after. That should be easy to implement even if Reolink does not provide a firmware inplementing AI alarms with ONVIF. How long does it take the alarm state to reset back to 0?

[ldwedari]

Great! Then since with ONVIF we can get a normal motion notification it should be possible to get the type of alarm by calling that GetAiState endpoint right after. That should be easy to implement even if Reolink does not provide a firmware inplementing AI alarms with ONVIF. How long does it take the alarm state to reset back to 0?

[jennettwheeler]

Just tested, that looks like a really great find! My 810As respond well!

However, the API is not happy with passwords that need percent encoding (e.g. # is %23). But that's fine because I can just make an API user with really long passwords.

Might be worth noting somewhere in readme?

If it helps, I have a slightly different response through:

[
   {
      "cmd" : "GetAiState",
      "code" : 0,
      "value" : {
         "channel" : 0,
         "people_state" : 0,
         "vehicle_state" : 0
      }
   }
]

Firmware: v3.0.0.177_21012103

[jennettwheeler]

Further to this, the GetAiState command does not exist on the NVR yet

[jrtraynor]

@ldwedari It doesn't stay as 1 for very long. Hard to tell walking in front of the camera and refreshing the link but 1 maybe 2 seconds max

[fwestenberg]

Great! Then since with ONVIF we can get a normal motion notification it should be possible to get the type of alarm by calling that GetAiState endpoint right after. That should be easy to implement even if Reolink does not provide a firmware inplementing AI alarms with ONVIF. How long does it take the alarm state to reset back to 0?

This is exactly what is done for NVR. At any motion, every camera is called to find out who triggered the onvif motion. So indeed, it is easy to implement the same for AI. I own a 820A, but had to update first to get the GetAiState API call supported.

[fwestenberg]

Please update the cam's firmware and check out the new release! The binary sensor should be updated with attributes!

[jrtraynor]

Hi,
Thanks for the quick update. Am getting an error with it though. Assume its connected with this so didn't log as s new bug.

Invalid config
The following integrations and platforms could not be set up:
reolink_dev.sensor
Please check your config and logs.

Logger: homeassistant.setup
Source: setup.py:280
First occurred: 10:07:32 (1 occurrences)
Last logged: 10:07:32

Unable to prepare setup for platform reolink_dev.sensor: Platform not found (cannot import name 'SensorEntity' from 'homeassistant.components.sensor' (/srv/homeassistant/lib/python3.8/site-packages/homeassistant/components/sensor/init.py)).

[jennettwheeler]

I also got that error. I also don't get any motion events. It always stays "Clear".

[fwestenberg]

The above error is unfortunately something with the Media browser for Reolink. Any other error in the log?

[jrtraynor]

Can't see any other errors in the log. Just tried a re-install to see if that brought up any others but no. Maybe unrelated but now when I go to config and change the stream to rtsp/sub and save it says "User input malformed" and doesn't save. So means the live view wont work.
I am seeing motion events in the log. No idea how to view attributes yet ( new to HA) but have to assume they are being set!

[jennettwheeler]

@jrtraynor I had to put something in the custom thumbnail path (I then went to the .storage/core.config_entries file to remove the custom thumbnail bit from the JSON).

Weirdly, I am not seeing any motion events at all... nothing in the log either. Don't know if this was an issue before as I only just set it up.

[wimpie007]

My 810A which showed motion events earlier ok, does strange things with this update
api.cgi?cmd=GetAiState works properly.
[ { "cmd" : "GetAiState", "code" : 0, "value" : { "channel" : 0, "people_state" : 0, **"vehicle_state" : 1** } } ]

How to see the vehicle detection? @fwestenberg ?? Binary sensor can only be on or off??

[macxq]

Guys, I'm owning 820A and little confused how checking for updates works. When i check in Reolink Client it was saying that I'm running the newest version from January, on the official page it is also referring to January version https://support.reolink.com/hc/en-us/articles/900004600266-21st-Jan-2021-Firmware-for-RLC-820A-RLC-810A.

But when i reach out to Reolink suport with the problem that "Car detection catchers cars outside of motion zones" they send my April firmware v3.0.0.251_21040912 with the fix https://support.reolink.com/attachments/token/RWjfZKeQp51lsSwq8vIG4BZlC/?name=IPC_523128M8MP.251_21040912.RLC-820A.IMX415.8MP.REOLINK.pak
Do you know if the is a way to stay up to date with the firmware. Definitely Reolink Client dose not do this.

[wimpie007]

@inzynierwatroba have noticed that too with my 810A.
I was on a version of EARLY march, and detection didnt work through API.
Searched on the website, and got a release from 21(?) march. Then API car/person detection works.
Clients also said 'camera up-to-date.'...

[macxq]

@wimpie007 where did you ware able to find the 21 March release for you camera on their web page ? I'm blind and just see https://support.reolink.com/hc/en-us/sections/360002374874-Firmware 21 Jan. version :/

[Morphy99]

Guys, I hate to break it to you, but the AI on Reolink sucks. I've tested with a 510A fed into Frigate for comparison and the Reolink misses tons of events. It's concerning the amount of tunable parameters in Frigate vs Reolink non existent AI settings. The sensitivity is up high too.

[wimpie007]

@inzynierwatroba oeps! i was wrong! i ment JANUARY release. (early january <> 21 january)
@Morphy99 i'm actually quite impressed with the AI... gives me TONS of less warnings. (altough not perfect)
Frigate looks nice BTW, but my RPI4 will not be fast enough to record al streams....

[Morphy99]

@inzynierwatroba oeps! i was wrong! i ment JANUARY release. (early january <> 21 january)

The latest firmware is here and it's March. For some reason they have a blog with articles which is top on google search and adds to confusion as they have a download area with more up to date and easier to find firmwares.

@Morphy99 i'm actually quite impressed with the AI... gives me TONS of less warnings. (altough not perfect)
Frigate looks nice BTW, but my RPI4 will not be fast enough to record al streams....

I had Frigate running on a dedicated RPi4 with 6 cameras, using a Google Coral and it hovered around 80% CPU. Trick is to use the low res streams for detection, which means smaller objects aren't picked up but it works for cars and persons.

I guess vs standard motion detection it works better yes, but then that was just so bad it was unusable for me. At least on outdoor cameras.

[wimpie007]

that page shows "15/3/2021", but if you download the file itself, it is the 21/01 release... (3.0.0.177)
Strange:)
Thanks for the info about Frigate. looks freaking nice :)

[wimpie007]

@Morphy99 what board did you use? USB Accelerator?

[Morphy99]

@Morphy99 what board did you use? USB Accelerator?

I used the USB Google Coral accelerator. I now have a m.2 key version inside a celeron intel nuc type box as I couldn't get the pi to work without an external powered hub and M.2 SSD + Coral. It looked a mess and didn't end up costing much less than an intel version in the end. The downside is the nuc box is now out of stock everywhere, I think end of life and next one up is quite a bit more. But that's another story! Have a look here for more info on Frigate.

Frigate is awesome. It's so accurate and picks up everything when tuned up correctly. It's still beta and needs some refinement but the future looks bright with a model learning GUI being worked on.

[jrtraynor]

Still trying to get this working. Can see the new attributes in the developer console but can't get an event to trigger. Either the script is wrong - very possible! or the attributes aren't getting updated. I also see there is an ongoing issue in HA between true/false and on/off and those in quotes that might not be helping. The main motion event is being triggered by the camera but attributes aren't logged so have no idea if they are being updated.

This is the script, just to update a date/time when a car is seen. Any pointers would be appreciated!

`alias: Back gate car
description: ''
trigger:

• type: motion
  platform: device
  device_id: 4f7f27a50356be1e8cff2d2525fd9bae
  entity_id: binary_sensor.back_gate_motion
  domain: binary_sensor
  condition:
• condition: state
  entity_id: binary_sensor.back_gate_motion
  attribute: vehicle
  state: 'true'
  action:
• service: input_datetime.set_datetime
  data_template:
  datetime: '{{ now().strftime(''%Y-%m-%d %H:%M:%S'') }}'
  entity_id: input_datetime.outside_car
  mode: single
  `
  (have tried state of on, 'on', true and 'true' )

[wimpie007]

mmm, i even can't see the new attributes...

[fwestenberg]

@wimpie007 please try to reinstall from HACS, next reboot HA. At least the attributes should be visible! And is your camera firmware up to date? Pls check manually at the Reolink site! My camera wasn't and also didn't show any update. But after manual update, AI worked!

[wimpie007]

@fwestenberg i'm on the latest firmware, and - as i mentioned earlier - the api for motion detection 'category' works:
api.cgi?cmd=GetAiState&user=admin&password=xxx shows
[ { "cmd" : "GetAiState", "code" : 0, "value" : { "channel" : 0, "people_state" : 0, "vehicle_state" : 0 } } ]
So the API most definately works.

Reinstalled en rebooted, but no change... RLC-810A...
even removed the camera from integration, and re-added it. No go.
no attributes appear.

[macxq]

@wimpie007 the same as you :(

[fwestenberg]

There are two files to check, at your config/custom_components/reolink_dev location. First the base.py file should contain this line:

reolink_dev/custom_components/reolink_dev/base.py

Line 177 in 6c8f3df

await self._api.get_ai_state()

Second the binary_sensor.py should contain this part:

reolink_dev/custom_components/reolink_dev/binary_sensor.py

Lines 120 to 129 in 6c8f3df

	if self._base.api.ai_state:
	for key, value in self._base.api.ai_state.items():
	if key == "channel":
	continue
	if self._state:
	attrs[key] = value == 1
	else:
	# Reset the AI values.
	attrs[key] = False

[wimpie007]

@fwestenberg
Yes, both files contain those line/parts. Exactly the same.

[wimpie007]

Update:
i updated HA to 2021.6.1 from 5.4(?) and strangely, all the cam entities where 'duplicated'!
for example:
binary_sensor.reolink3_voordeur_motion (old one) now has a brother:
binary_sensor.reolink3_voordeur_motion_2

The good news: reolink3_voordeur_motion_2 has the new attributes!

[fwestenberg]

It sounds like you were still running some older version of the Reolink integration also. Just remove the cam, so all entities are gone. Next add it again and it should be fixed.

[wimpie007]

i definately had the latest reolink integration files in place. Checked the code.
For some reason, the new integration didn't work... (did mutiple reboots)
It was only after the upgrade of HA itself that is started to work.
I renamed the instances and now, everything works flawless, detection works nice.
Thanks for the great work! @fwestenberg Much appreciated.

[PhilipWhiteside]

I'm having trouble getting the Person Detection entity within HA. I have attempted the following:

• Removing the Reolink integration, rebooting, adding back, re-adding camera
• Forced new entity by changing camera name, IP, and credentials
• Factory reset the camera
• Downgraded the camera
• Fresh HA install (see next comment)

With versions:

• Home Assistant version: core-2021.6.6
• Reolink camera model: RLC-820A
• Camera firmware number: v3.0.0.251_21040912, also tried 250_21040801
• Reolink_Dev: v0.17
• Home Assistant supervisor: supervisor-2021.06.6
• Home Assistant OS: 6.1

Confirmation of person detection support via http://camera-01.lan/api.cgi?cmd=GetAiState&user=admin&password=reolink

[ { "cmd" : "GetAiState", "code" : 0, "value" : { "channel" : 0, "face" : { "alarm_state" : 0, "support" : 0 }, "people" : { "alarm_state" : 0, "support" : 1 }, "vehicle" : { "alarm_state" : 0, "support" : 1 } } } ] 

When I'm in the camera feed:

[ { "cmd" : "GetAiState", "code" : 0, "value" : { "channel" : 0, "face" : { "alarm_state" : 0, "support" : 0 }, "people" : { "alarm_state" : 1, "support" : 1 }, "vehicle" : { "alarm_state" : 0, "support" : 1 } } } ] 

Confirmation of latest release with latest commit:

reolink_dev $ cat base.py | grep get_ai_state
        await self._api.get_ai_state()

I am a bit at a loss of what to try next.

[PhilipWhiteside]

As a further test I have just deployed a brand new HomeAssistant using the container on a different VM (my main HA is the VM OS), manually added reolink_dev (not HACS), and added the camera. I get the same result, only 7 entities, no Person Detection.

I feel like I'm missing something very simple as others above seem to have it working without much effort. I just don't know what that is, or have any error to look into, etc.

[wimpie007]

Motion detection is an attribute of the motion sensor:

Did you look in the right place?
I had problems to get it working also. Rebooting didn't help. After upgrading HA, suddenly the attributes appeared. Never new why...

[PhilipWhiteside]

Hey @wimpie007 thank you for that.

Perhaps I don't fully understand, I do have what you show, but the "People state" is always False. On the /api.cgi?cmd=GetAiState page it does show specifically person detected with "people" : { "alarm_state" : 1. As the attribute of "People state" does not change I cannot create an automation for person detection specifically.

When looking at /developer-tools/state I see the state change from off to on but people is always false. Do I have to do anything else for that attribute to be updated?

[jrtraynor]

Hi @PhilipWhiteside ,
What you are seeing is the same as I see. Have not yet seen the attributes change in HA even though the json from the camera has the correct data. Is there some way of getting a detailed log of what is going on, might help pinpoint what is (not) happening.

[PhilipWhiteside]

Hey @jrtraynor the only logs I've seen are the HA ones (Supervisor > System > Logs), or when running the container (docker logs -f <container-name>). I don't think that the JSON you see on the camera API is what is sent.

Further up in this issue is discussions of NVR checking each cam for which one detected motion etc. I don't fully get it as I don't have an NVR unless it's referring to this project itself, so I could be wrong here, but it seems like a motion event is detected by the binary_sensor, then this integration goes out and queries the camera for more information, updating the attributes. It is not a push from the camera to HA in a single event.

Also, as a binary sensor, it can't have more than 1 state, so it triggers true or false, the attributes are not the sensor itself? If this is the case, then I may be leaning more towards the suggestion above of Frigate with a coral (just bought the PCIe one to try), and returning the Reolink camera.

[domosgabor]

Hi, first of all thanks for this great integration!
Does the person/vehicle detection work with an NVR? I've connected my Reolink 1220A to my NVR and the motion sensor has no attributes in HA. If I connect the camera diretly to my POE switch I can get the GetAIState http responses but if it's connected to the NVR I receive this Json message: [ { "cmd" : "Unknown", "code" : 1, "error" : { "detail" : "not support", "rspCode" : -9 } } ]
I suppose that it's still not supported by the NVR. Is it correct? If ye, is there any workaround solution?

[rodakswe]

Hey @wimpie007 thank you for that.

Perhaps I don't fully understand, I do have what you show, but the "People state" is always False. On the /api.cgi?cmd=GetAiState page it does show specifically person detected with "people" : { "alarm_state" : 1. As the attribute of "People state" does not change I cannot create an automation for person detection specifically.

When looking at /developer-tools/state I see the state change from off to on but people is always false. Do I have to do anything else for that attribute to be updated?

Did you ever figure this out? I have the same problem with RLC-822A firmware version V3.0.0.250.
"People" never changes to "True"
Prior firmware update (V3.0.0.177) of the camera I had it working and the available attributes was:
"People state"
"Vehicle state"
The "People state" changed to "True" as it should.
After firmware update (V3.0.0.250) the attributes changed to:
"Face"
"People"
"Vehicle"
Non of them ever gets "True" even if the motion is "Detected"

[freeload101]

API is not happy with passwords that need percent encoding (e.g. # is %23).
You can use curl to parse that for you :

export key=curl -ks https://dc2plxssh003.newellrubbermaid.com:8089/services/auth/login --data-urlencode username=myusername --data-urlencode password="mypsassowrd***!" | grep sessionKey | sed -r 's/<sessionKey>(.*)<\/sessionKey>/\1/g'|sed 's/ //g'

[smenzer]

I just updated the firmware on my NVR and cameras and now the api seems to have disappeared ... I now get redirected to a web ui where I can log in and view the nvr/camera... anyone know if they fully removed the API or if it just needs to be accessed differently now?

[smenzer]

ah... seems like the new firmware requires https and not just http ... that solved. phew!