Snapshot Images

[wehbs]

I noticed when adding in the gif logic that an event may sometimes only generate one image snapshot. Is it possible to force a minimum of like maybe 3 snapshots or what in the logic today determines how many snapshots to gather from an event?

[chiefMarlin]

The number of snapshots id directly related to the number of objects that are detected, if an object is detected in a frame and this object has not been detected before (in the same place and of similar size) then an image (snapshot) of that object is saved.
So assuming that the purpose of gif is to provide a tiny clip instead of an image, it may not be possible to use snapshots to generate a proper gif as you would need surrounding frames to show fluid motion ? Maybe its better to generate a gif from the whole video clip ?

[wehbs]

I see, and so these are grouped together based on the Event Gap preference?

[chiefMarlin]

Exactly
So every time a new motion is detected, a new motion id is generated and lasts for a minimum of event gap time

[chiefMarlin]

What did you have in mind to include in the gif ?

[wehbs]

Well the main goal is to have a quick glimpse of the event that caused the motion in a push notification similar to how Nest cams do it. However, speed is very important to try and get the push notification to the client asap. I figured the fastest way to go about it would be to just stitch the snapshots in sequential order that you are already generating as opposed to having to decode video. So I think I have 2 options:

1. If the event is just a single object then we push just a still image and if it's multiple then we create the gif and push that.
2. Decode the low res video and find the perfect balance between decent quality, file size, and decode time for the gif creation.

Unless you have any other thoughts?

[wehbs]

Or actually completely scrap the gif idea and instead send a push of each single snapshot but cropped into the object box. I would prefer this actually overall. What do you think?

[chiefMarlin]

Either is possible, for gif we could store every Nth frame whether there is an object or not in the frame to make a fluid gif.
One downside to sending alerts the moment object is detected is that they may spam the user.
What do you think ?

[wehbs]

something like this may be best?

[wehbs]

Either is possible, for gif we could store every Nth frame whether there is an object or not in the frame to make a fluid gif. One downside to sending alerts the moment object is detected is that they may spam the user. What do you think ?

Yea, that's always a risk but should be able to be mitigated through Event Gap, non detection zones, what objects to detect, etc.

[chiefMarlin]

Yeah, good idea.
Which service did you want to use ?
Theres pushover, ntfy.sh
Theres also slack

[wehbs]

So I went through a few services recently in a small little project I was working on and found that for the moment pushover is the best. I would love to use ntfy.sh but the iOS support is sparse and at the moment does not support images or gifs only text unless you're on Android. But I think ultimately it'd be cool to add rich notification support for multiple services.

[chiefMarlin]

can you send images with pushover or it requires a url for image ?

[wehbs]

You can send the image directly, and according to them they delete from their server as soon as the client device downloads it.

[chiefMarlin]

Alright, ill implement it in the coming days

[wehbs]

I see in the code where you create the object rectangle and save the image, I'm thinking I can save a secondary image there that uses the same coordinates to create the cropped version with no rectangle or label. I can try to implement along with a pushover function and pushover key and token parameters in the json. If it's ok with you I'll take a stab at it.

[chiefMarlin]

I think it would be better to just push the cropped frame directly to pushover and avoid saving anything to file as there is no need for it.
I think this should be a fairly quick thing to add, let me push the changes and you can see if there is anything you want to modify from there

[wehbs]

ah you're right even better. Ok, sounds good. I'll hang tight, thanks!

[chiefMarlin]

Ok, its ready, if you grab the latest code you can give it a try.
However im not sure if sending notification on every single object detect will be a good idea, it basically flooding you with notifications.
I'll play around with it to see what works best and let you know what i come up with

[wehbs]

Ok, tested. Notifications lightning fast. However, it's still the full image, would it be easy to crop the image that gets pushed to the dimensions of the object rectangle? I also see what you mean about the push spamming. Perhaps notifications needs it own preference setting for what objects to notify on and not just the same objects that are chosen to be tracked globally.

[chiefMarlin]

I also looked into cropping the object, however we need to keep in mind that there is a high chance of having multiple objects in the frame. Would that result its 2, 3.. images being sent ? All n all im still thinking of about the best approach here.
I think it would be beneficial to think about a few points, and hopefully that will guide this in the right direction.
Would it make a difference whether we get notification right that second or in 10 seconds ?

[wehbs]

ah I understand, what if we only sent the image of the first object that triggered motion detection, this would ensure no other image for the remainder of the event gap.

If that particular frame contains more than 1 object then we zoom the crop out until it contains all the cropped objects in frame.

[chiefMarlin]

That could work, but if the importance here is to get notified about objects in real time, only seeing the first object and not getting any more notifications until later, wouldnt that make one think that there has been no more motion since first object ?

[wehbs]

I suppose we could add a secondary event gap option specific to push notifications and recommend a lower value like 5s?

[chiefMarlin]

What about a collage ?

[wehbs]

Oh this is cool. I think if these were cropped it could work well. But then you're left waiting longer for first push notification on initial motion.

[chiefMarlin]

With a collage we could potentially wait for half the gap time (say 10s) and put all existing objects into collage and send it over, and either repeat that every 10s or send another final alert at the end of motion with a video ? or gif

[chiefMarlin]

Alright, then how about for v1 we send initial notification on first object, then send a lightweight video on motion end ?

[wehbs]

Oh I love that, pushover doesn't support video, but it could be a gif?

[chiefMarlin]

Yeah, let me try with a gif

[chiefMarlin]

[wehbs]

GIF looks good, and size is great.

[chiefMarlin]

So i just added sending first notification as soon as object is detected, it will send another alert when motion ends

[wehbs]

Sweet, going to check it out now!

[chiefMarlin]

What do you think so far ?