Object classification with Frigate+ #19117
Replies: 3 comments 8 replies
|
This is related to this conversation about label hierarchy. |
|
I believe otter is a candidate label, so realistically if you tell the model it is not a cat but an otter you will temporarily be training the model that it is nothing there, but in general I would still expect this to be labeled as one of the animals.
I don't think that really makes sense, it is not a sub label in the same way that fedex / ups is. Fedex / UPS require detecting a separate specific attribute of the car (a logo). So when you suggest the same thing for an animal (let's say dog) you are essentially saying that the model should detect the same thing as an animal but also know it is a dog. If this is not how it would work then we would require a secondary classification model, which is an additional resource user and increases the latency, which is a regression for many users. That does not work in the same way. Realistically, if all you care about is that something is an animal then you can just change your labelmap to call all animals
I don't think that is always the case, for example I have a dog so a dog in the backyard is normal. But if a rabbit or cat is in the backyard then I can't let my dog out as he will likely chase them / potentially try to get out of the yard. This also further becomes problematic as people want to recognize if a cat is their cat, and Frigate 0.17 has had object classification implemented. If everything was just
This is already how Frigate works, for an object to be considered a potential true positive it must have had a position change. |
I didn't see otter in the list, cool though if it is on the candidate list or ends up on the real list. From your response, I infer that if tell the model that a given image is not a cat (or other known thing), and I either label it as a candidate label, or don't label it as anything, I am going to tell the model to ignore things that look like my image. Therefore, I should mis-label unknown or unofficial creatures as a currently supported animal (and when the the the correct animal becomes supported, update my labels.)
Perhaps "sub label" isn't the right terminology, my goal is to ensure that unknown or unsupported animals cause events to be created. It sounds like I need to ensure that objects I care about are mapped to some known object type. My worry is polluting the model for others.
But can I specify how far an object must move? I have cases where swaying branches create enough of a position change to generate an event. If I could say that the position change must be 20 percent of the frame, it would eliminate those false positives. I could mask out trees and bushes where I don't expect cars or people, but those trees and bushes can easily be behind where cars and people might be, and maintaining those masks per object type is a bunch of work. Being able to specify a motion threshold in terms of distance would be useful. |
@blakeblackshear can respond more here, but I'm pretty sure he reviews every single image before including it in the base model. |
|
Yea. Images aren't included in the base without a review, so you can't pollute the model for others. |
In many cases this isn't as much as you would think. False positive objects often get larger and smaller which appears like a change. Keep in mind when an object gets closer to the camera it really doesn't change position much, it only gets larger and the bottom changes a bit. This would lead to a problem of false positives more often than you may think. |
|
I have had to start researching some of the different animal families across the world from all these wildlife requests. Turns out that otters and badgers are both mustelids. I may end up converting the badger label to mustelid, so I think its best to label them as badgers. I have also had requests for marten, polecat, and weasel. All of these are mustelids. |
|
At some point it might make sense to have the fast, initial detection trained down to, say, the order level of the taxonomic hierarchy based on general external morphology and then pass the detection on to a bigger, slower model that goes deeper in the hierarchy, ideally all the way down to species level (or unique individuals for pets), similarly to how birds are already handled. |
This is already possible with the under development custom classification model feature in 0.17, I already have a model working well classifying my dog vs other dogs |
|
I have a question a bit related to this. I tried yesterday a new yolonas9 model, and it recognizes raccoons very well. Despite the fact that here (in Finland) they are actually badgers, not raccoons. Should I start (keep) reporting them as false positives and mark them as badgers instead? Or should I be happy that it recognizes them as something, and does not just skip them? |
|
I believe badgers are a candidate label, so you should enable badger in your Frigate+ camera settings and then label them as badgers |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
I have a Frigate installation that has several cameras at a vacation house. The house is unoccupied most of the time and my primary use case is knowing what happens there, particularly in terms of what wildlife comes by.
Cougars live in the area and I would love to get a picture of one. The place is on the water, and I get otters in the yard occasionally.
I would like Frigate to create an event (give me something to review) when any animal, without regard to what kind of animal comes along.
With this in mind, when I am reviewing events and submitting images to Frigate+, should I provide negative feedback and should the decision depend on if the correct model exists?
For example, I assume if Frigate sees a dog as a cat and I say “No that’s not a cat, it’s a dog” that in the future if a cougar comes along, I won’t have reduced that chances of Frigate catching the event. Frigate may still think the cougar is a dog, but it will at least report an event, since it knows about both cats and dogs. However in the case that Frigate thinks an otter is a cat, and I say “No, that’s not an cat”, but since otter isn’t a known object, by reporting a false positive and not reporting the correct positive, am I reducing the chances that an event of any kind will be reported the next time an otter comes along?
If I don’t want to miss otters, should I
As a suggestion / feature request I would prefer to have a single master object called “Animal” with sub labels for kinds of animals. Just like how we have “Car” with sub classifications of Ups and FedEx, I would like to see “Animal” with sub classifications of “Cat, Dog, Deer, etc.” From a security system point of view, I would think that most people primarily care about if the event was a Person, Vehicle, or an Animal and would rather be assured that any vehicle will raise a vehicle alert, and that any animal will raise an animal alert – even if the specific vehicle or animal type is not known. Most commercial solutions seem to focus on People, Vehicles, and Animals as general objects.
As another suggestion / feature request to support my use case, I would like to have an event filter for “object moved percentage” which I could apply to any object and specify that for an event to be created, the detected object needs to move x percent through the image frame. While I can create masks and I can increase my object detection thresholds, I still get lots of false positives caused by branches or water moving, shadows, etc. If I set my thresholds too high, I will miss events. In almost all cases however, my false positives are of the mostly static background image. Anything I care about will be moving and moving significantly. It would be a killer feature if I could say “I don’t care what you think that object is, but if you see a set of pixels that you think is any recognizable object, and that set of pixels moves a third of the way through the frame, raise an event.”
All reactions