[Feature request] Area detector

[BarcaBear]

Could an area detector be added? And output the position via the API. This would allow one to count and output the direction. As well as Output the location within the area detector.

[tdurand]

Hi !

Thanks for this, could you detail a bit more the use case you are looking to solve ? not sure I understand it fully.

[BarcaBear]

If detection is required in a specific area of the image,

Use case 1 pedestrian detection for loitering. An area would be created to only output via the API in this area.

Use case 2 Vehicle break down. Draw an area detector in a lane and output via the API. The higher level system would deal with what to do.

Use case 3 Bicycle priority. Detect in an area if a bicycle is present and output via the API. The higher level system would deal with what to do.

[tdurand]

Thanks ! Yes I see this could be a handy feature to add in the future.. We also had some report from users telling us that it would be nice to just restrict the tracking to some part of the frame for privacy issues.

[shams3049]

Hi,

I wanted to clear up some points regarding the area detector feature (mentioned here).

I know you are busy with the v3 development. I can take up the task of development of this feature.

I you have time, please do walk me though how would you @tdurand develop this? in the the current project. I want to know if you see it as an extension feature of the current tracker logic, or any particular pointers I should consider.

[tdurand]

Hi @shams3049

Honestly I didn't think it over.. What are the use case you are looking to solve with something like this ? In the best of the world how do you see this feature ?

[shams3049]

One of the use case that it solves for our project at harbor: sometimes there are parking spots next to the road, if we draw a polygon, we can then keep a counter of cars or truck inside the polygon, I was of the mind that we can generate data on how long the parking is occupied and around which times (for example when was the polygon object count turned from zero to one and then zero again). This data in our case is really helpful to the port authorities.

Brainstorming:
If its possible to extend the way you implemented drawing the detector lines to form a close polygones. Your implementation of barometric calculation can then be used to make a counter for the area (counter going higher when objects enter the area and viseversa).

[tdurand]

I see, I will get back to you next week on some thoughts I have on this.. right now trying to wrap up stuff for the week-end ! Have a nice one ..

[BarcaBear]

I have a very rough alpha version that allows polygons to be drawn. If you think it would be helpful I can copy here for you to look at?

[shams3049]

I have a very rough alpha version that allows polygons to be drawn. If you think it would be helpful I can copy here for you to look at?

Yeah please do, could be helpfull.

I see, I will get back to you next week on some thoughts I have on this.. right now trying to wrap up stuff for the week-end ! Have a nice one ..

No hurry, once you have time, we can also discuss over zoom or something. Have a nice weekend.

[BarcaBear]

You can get the area detector from this link https://www.dropbox.com/s/38fgtfowxne9rc4/Area%20detector%202.rar?dl=0

[tdurand]

@BarcaBear Thanks for sharing, could you showcase us what you have built with a few screenshot or a quick demo screencast ?

@shams3049

I see, it is for sure possible but not that straightforward to implement.. I guess we can think of it as a Counting Area and not a Counting Line and we would need to

• Add another mode to the editor so we can draw polygons, with all the overhead that it adds UX wise because right now we rely on the simplify of having only one type of things we can draw
• Create the corresponding logic to "count" , one straightforward way to do it would be to tell from the coordinates if an object is inside or outside at every frame.. but this won't be very reliable on the long run because ID switching can happen.. so maybe better to just count when it crosses edges of the polygon.
• Add on the Data view another type of data which would be the "Areas" along with "Counter" ..
• Also this wouldn't cover the use case of beeing able to ignore certain areas for privacy issue.. So we would need to have another mode where we can draw areas that won't be tracked.

not easy to limit the scope of this feature

[BarcaBear]

You can see from these two videos how it is currently working

https://www.dropbox.com/s/vk52ryo1xov2eps/Screencast%20from%2006-05-20%2011%3A38%3A05.webm?dl=0

Drawing multiple area detectors.

Issue 1. Only the last drawn area detector changes the bounding box colour.
Issue 2. The delete icons are displayed on the top left corner. If you change to tracker and then back to area. The delete icons are displayed in the correct location. Bottom left of detector

My comments:-

• Add another mode to the editor so we can draw polygons, with all the overhead that it adds UX wise because right now we rely on the simplify of having only one type of things we can draw

BarcaBear- Ideally there would be one location that could allow both a count and area detector to be drawn. Selecting with a button which to draw. You can see in the videos that there is a "count" and "area" option. These two could be one.

• Create the corresponding logic to "count" , one straightforward way to do it would be to tell from the coordinates if an object is inside or outside at every frame.. but this won't be very reliable on the long run because ID switching can happen.. so maybe better to just count when it crosses edges of the polygon.

Barcabear- My thoughts are to have the count detector stay as a count detector. And the area detector output tracking information as does today. But add an ID for the area detector. For example below:- The count detector could be drawn on top of the area dteector/ or bottom.

"trackerDataForLastFrame": {
"frameIndex": 4646,
"data": [
{
"id": 5,
"detector_id". 101,
"x": 340,
"y": 237,
"w": 60,
"h": 45,
"bearing": 103,
"name": "car",
"countingDeltas": {
"94afa4f8-1d24-4011-a481-ad3036e959b4": 349.8589833356673
}
},

• Add on the Data view another type of data which would be the "Areas" along with "Counter" ..

Barcabear-- Yes, as i suggest above.

• Also this wouldn't cover the use case of beeing able to ignore certain areas for privacy issue.. So we would need to have another mode where we can draw areas that won't be tracked.

Barcabear- https://www.dropbox.com/s/xygvbpd6s2e537y/Screencast%20from%2006-05-20%2012%3A01%3A01.webm?dl=0

If we use the polygon to detect/track only inside the polygon. Would this not take care of ignoring certain areas for privacy issues?

[shams3049]

@BarcaBear Nice work, I see detection inside the polygone. do you have implementation for the the count inside the polygon?

[BarcaBear]

No, I would use the count detector for that. And the area detector for tracking information.

[shams3049]

@tdurand @BarcaBear's solution looks like a good start, what do you think, if we pull it to current opendatacam and work on it further to reach a polished version.

[tdurand]

I just looked at the videos, if I summarize the current state of the work of @BarcaBear :

• Adds another mode "area" , which enables to draw polygons.
• When recording, this enhance the tracker data with id of the area for all the bbox center that are in the area for each frame
• No changes to the Data view panel
• No changes to the counting logic

I kind of like this minimal approach to this.... we could add something to restrict the tracker to only the polygons drawned and we would also have the "privacy" feature.

@BarcaBear Could you do a pull request so we can look into polishing/improving the implementation ? (I didn't downloaded / looked into the code yet)

[shams3049]

I think from the usability, I think we can add drawing polygone next to drawing lines and present it with a different icon.

[tdurand]

I would actually lean towards keeping this as an "extra" mode because it is actually a different concept from the counting. You can have counting lines and polygons areas... So in my opinion it is much clearer to have Areas as another distinct mode.

In the minimalistict approach those polygons won't enable you to "count" stuff... so maybe also this doesn't solve your use case @shams3049 ..

[shams3049]

@tdurand I think it's fine with me as well to approach it as distinct node. we can see how much counting logic can be integrating for the use case, in any case I can always look for a higher level solution using the API.

Having the polygone integrated into opendatacam, definitely sounds like feature that will enable new use cases.

[tdurand]

yes this is how I see it also. We don't want to bloat OpenDataCam with lots of features we need to implement and support / maintain / improve in the future.

I think this minimalistic approach makes a lot of sense, because it uses the "power" of the nice UI editor of OpenDataCam to help people define easily areas and adds minimal logic to enrich the API with them.

I see lots of benefits also to be able to say that we can restrict the tracker to a certain part of the frame for privacy issues also.

[rantgithub]

If I may give my input here

For some similar use/request I think the area is more suitable for detection/messaging/alert.

Assuming the parking example, you want to know if there is a car or not there, and if something is on that area you can get a message/email/etc.

you can emulate a "zone" by putting different lines around the area.

I think the zone will be very useful, for these kind of scenarios and you can mixed/used in parallel with the counting lines.

Meaning and object entering into the zone (just that for this feature) and you add a counting line inside the zone for that to be counted

In this way you minimize changes and maintenance on the code and can keep the current setup/code.

[BarcaBear]

Could you do a pull request so we can look into polishing/improving the implementation ? (I didn't downloaded / looked into the code yet

BarcaBear- I'm not familiar on how to do this here, and don't have the time to learn how to at the moment.

If someone is more familiar than me and can use the code I sent the link to. Then please go ahead.

[shams3049]

@BarcaBear I can do it.

[BarcaBear]

@BarcaBear I can do it.

Have you made any progress on this shams3049?

[shams3049]

@BarcaBear there are some conflicts with the current branch, Didn't had to time to resolve it. its good that you reminded me here, I'll get on it this Saturday.

[tdurand]

FYI

@rantgithub funded me 💰️ to work on this.. I did something a a bit different that the one specified here, which enables to count things in polygons and not only enhance the tracker with wheter the object is inside a zone or not.

• Added two buttons on the counter editor to draw either a line or a polygon
• By default for polygons the counter will only count the object if it is inside the zone AND if it just crossed one the edges of the polygon (like a counting line).. this is done to avoid counting twice an object if it changes ID inside the zone..
• Enhance tracker data with the area the object is inside for each frame
• Added parameters to customize the algorithm behaviour in the config.json: https://github.com/opendatacam/opendatacam/blob/development/documentation/CONFIG.md#counter-settings

See implementation here: #237 . BTW I won't do any release of OpenDataCam with this feature so you need to pull the development to use it for now.

I'm leaving this open as in the future we might want to have a separated "Area" tab like @BarcaBear to use areas not for counting but for filtering the objects for privacy reasons.. let's see.

[b-g]

Yay! Very cool!

[BarcaBear]

Great work. I'm getting an error when trying to run in demo mode

"Config.json version doesn't match Opendatacam package.json"

Any ideas why?

[tdurand]

Make sure the version in the package.json is the same as the one in config.json .. If you use the development branch it should be the case

[BarcaBear]

I did check that and they match. The master branch is working fine. I will test it in a tx2 rather on my dev machine.

[tdurand]

hum.. strange.. you can uncomment this line if really it does not work: https://github.com/opendatacam/opendatacam/blob/development/server.js#L21

[BarcaBear]

Commenting that line out worked.

[tdurand]

this is implemented on development branch, thanks to @rantgithub funding last year .. will be part of the upcoming v4 release