Option to save/publish via MQTT snapshots of all identified targets even when event is in progress

[raintonr]

Describe what you are trying to accomplish and why in non technical terms

From what I can tell this is how things currently work (0.12.0-da3e197):

1. When a new target is seen and no event is in progress for this camera (or possibly this is for the combination of camera+target type - am not sure) then an event is started and a snapshot taken and published to MQTT if that has been configured.

2. If that target stays in frame an event is deemed 'In progress' so if a second target of the same type comes into frame then a snapshot of that second target is not taken or published.

Describe the solution you'd like

An option to enable saving/MQTT publishing of snapshots of all identified targets (passing the relevant threshold of course), no matter if an event is in progress or not.

Guess this means an event in the UI could have multiple snapshots associated with it and therefore the UI needs some way to view thumbs of these and/or scroll through them.

[NickM-27]

This isn't correct. An "event" is really just a tracked object. There are separate events for each tracked object. Frigate does not limit snapshot saving for objects of the same type. A snapshot and thumbnail are saved for each object regardless of in progress or not.

[raintonr]

OK - thanks for the clarification. I was not seeing that but perhaps there is something fishy going on with my MQTT broker causing snapshots to get lost then. Looking into that so will close this issue.

[NickM-27]

I think you're confusing some things. The image sent via MQTT on the snapshot topic #2199 isn't meant to be per-event afaik.

If you want to access the event snapshots you need to use the event topic and use the event_id to retrieve the snapshot

[raintonr]

Ahhhhhh... so I think what you are telling me is that snapshots in the UI, for example this...

... are NOT the same as those that will have been published to MQTT.

For this example I can see that the snapshot comes from http://frigate/api/events/1682423523.088004-s7026i/snapshot.jpg. So to get all the newly tracked targets what I should really be doing is watching the frigate/events topic and for every change just go fetch the relevant snapshot.

Cool - think that will work.

And I think you are saying that each new target causes a new event. In that case, if two people walk into the field of view of a camera a few seconds apart, an event will be generated for each person. And therefore if one is watching the events topic and sending snapshots for that event somewhere you'll end up with two images in this scenario - one for each person. That would be exactly what I'm looking for so thank you 😄

Is it important to note anything from the frigate/events topic or sufficient just to grab the ID and construct the snapshot URL to fetch directly from that?

[NickM-27]

Ahhhhhh... so I think what you are telling me is that snapshots in the UI, for example this...

... are NOT the same as those that will have been published to MQTT.

that is correct

For this example I can see that the snapshot comes from http://frigate/api/events/1682423523.088004-s7026i/snapshot.jpg. So to get all the newly tracked targets what I should really be doing is watching the frigate/events topic and for every change just go fetch the relevant snapshot.

Cool - think that will work.

👍

And I think you are saying that each new target causes a new event. In that case, if two people walk into the field of view of a camera a few seconds apart, an event will be generated for each person. And therefore if one is watching the events topic and sending snapshots for that event somewhere you'll end up with two images in this scenario - one for each person. That would be exactly what I'm looking for so thank you 😄

That is correct, however you should be aware that frigate has a cleanup logic to cleanup events with the same label that happened in the same timespan on the same camera. So if two people walk by and create two events that may be merged into one event.

Is it important to note anything from the frigate/events topic or sufficient just to grab the ID and construct the snapshot URL to fetch directly from that?

Depends on your usecase, there is a lot of information available there

[raintonr]

So I quickly wrote a script to pull event IDs from the frigate/events topic, fetch the snapshot based on that and send to myself. Works... kindof:

1. Sometimes this is just the frame straight from camera. No bounding box is drawn on the target (even though the box array is present in the message).
2. Sometimes a bounding box is drawn.
3. Sometimes a 404 error is returned.

Is there any way to know when the snapshot with the bounding box has been prepared and only then fetch it? Looking at the published message I don't see any obvious way to do this.

Basically, of the 3 scenarios above I'm only interested in 2. I don't see any obvious way to determine when to fetch the snapshot in order to guarantee this though.

I do note the 404 is transitory because the frigate/events topic seems to get numerous quick updates and if the fetch after one returns the 404 error soon after another message is published after which the snapshot was successfully returned. So sure, one can build an automatic retry but should that really be necessary? So should we consider this 404 error a bug? I mean, once an event has started surely it's snapshot should always be available?

Another question arising from this is that it seems that the snapshot image for an event is recreated multiple times - why is this? I guess that a target's score might cross a threshold which starts the event, but then the target might move and it's score might rise, in which case perhaps it would be better to re-create the snapshot from that newer 'higher score' frame. That is fair enough, but why does the snapshot sometimes have a box drawn on the target and sometimes not?

[NickM-27]

So I quickly wrote a script to pull event IDs from the frigate/events topic, fetch the snapshot based on that and send to myself. Works... kindof:

1. Sometimes this is just the frame straight from camera. No bounding box is drawn on the target (even though the box array is present in the message).

If the event is in progress you have to specify bbox=1 as a query parameter

Another question arising from this is that it seems that the snapshot image for an event is recreated multiple times - why is this? I guess that a target's score might cross a threshold which starts the event, but then the target might move and it's score might rise, in which case perhaps it would be better to re-create the snapshot from that newer 'higher score' frame. That is fair enough, but why does the snapshot sometimes have a box drawn on the target and sometimes not?

yes a new snapshot may be saved, this is based on the best_image_timeout and some other factors. As far as the box behavior, see my answer above

[raintonr]

bbox=1 always for me then. All good 👍

Thank you 😃