"area" in annotations

[nico00]

Hi,
I'm creating my own dataset but writing the annotations I've found a field called "area" that I didn't understand.
According to my analysis, it doesn't refer to:

• image area (width x height)
• bounding box area (width x height)
• segmentation area

Any idea about how I should calculate it ?
Thanks
Nico

[tylin]

It means segmentation area.
You can use mask.area function to compute it.
https://github.com/pdollar/coco/blob/master/PythonAPI/pycocotools/mask.py#L35

[nico00]

Thanks tylin.
As I have a polygon that describes the segmentation, to get the area value I have to:

• convert the polygon to a mask, with MaskApi.frPoly (I'm using LUA)
• encode the mask to RLE, with MaskApi.encode
• get the area, passing the RLE to MaskApi.area

Am I right ?
Thanks
Nico

[tylin]

That sounds right to me.

[nico00]

Hi tylin,
sorry to bother you but I've not figured out how to solve this problem.

Basically I've to create an annotations JSON, structured in this way:
{
"annotations": [
{
"area": ???,
"bbox": [ 199.84, 200.46, 77.71, 70.88],
"category_id": 1,
"id": 156,
"image_id": 558840,
"iscrowd": 0,
"segmentation": [ [ 239.97, 260.24, ..., 228.87, 271.34 ] ]
},
...

The only value that I miss is area. As you suggested I'm trying to calculate it using MaskApi.area
This is what I wrote, just to give a try:

local coco = require 'coco'
maskApi = coco.MaskApi
polS = { torch.Tensor( { 239.97,  260.24,  222.04,  270.49,  199.84,  253.41,  213.5,  227.79,  259.62,  200.46,  274.13,  202.17,  277.55,  210.71,  249.37,  253.41,  237.41,  264.51,  242.54,  261.95,  228.87,  271.34  } ) }

local h = 427
local w = 640
local R = maskApi.frPoly(polS, h, w)
local Rs = maskApi.encode(R)  -- returns an error: attempt to call method 'size' 
local area = maskApi.area(Rs)
print (area)

maskApi.encode(R) returns "attempt to call method 'size'". Looking at the code of that function is seems that R has to be a 3D tensor but in my case is 2D.

I'm becoming crazy. Any suggestion?
Thanks
Nico

[tylin]

One segment can contain multiple polygons {poly1, poly2, ..., polyn}, I think that's the reason to cause error.
You can load COCO annotations and print the polygon format in Torch to gain more understanding.

[nico00]

It seems that the problem was the sequence of calls. The right sequence should be:

local R = maskApi.frPoly(polS, h, w)
local area = maskApi.area(R)

The only detail I've noticed is that the area value I got, using LUA, is slightly different than the value I expected, according to some annotations you created (instance_val2014.json), for instance:

expected result: 2765.1486500000005
actual result: 2760

expected result: 1545.4213000000007
actual result: 1542

Does this mean my approach is wrong ? Or is this a consequence of using LUA instead of Python ?
More important, will this cause problems during the training session ?
Thanks
Nico

[sifengyang]

@tylin
Have you @nico00 solved the problem of 'area'? I experiment with your method. my code is this frPyObjects = _mask.frPyObjects segmentation = [[51.69, 292.27, 49.77, 241.33, 58.18, 240.13, 58.42, 292.03, 52.65, 292.51]] rs = frPyObjects( segmentation, 480, 640 )
output
[{'counts': 'a_o0d0\\c0l0TO4LO1000O100``W8', 'size': [640L, 480L]}]
How can I solve this mistake?

[AbuBakrCh]

@sifengyang This is not a mistake. After this you have to do:
area = _mask.area
a = area(rs)

[hazirbas]

those who need to write their own .json files can check my implementation for Davis 2016 benchmark:

coco-json-converter

[IaroslavS]

I hate RLE format. Someone wanted to make data lighter with the cost of lots of problems to other people.

[Johnqczhang]

It means segmentation area.
You can use mask.area function to compute it.
https://github.com/pdollar/coco/blob/master/PythonAPI/pycocotools/mask.py#L35

Hi @tylin, it seems that the "area" field of the prediction is given by the box area for the evaluation of object detection and keypoints detection task, according to line-338 and line-357 in the code snippet below:

cocoapi/PythonAPI/pycocotools/coco.py

Lines 331 to 359 in 8c9bcc3

	elif 'bbox' in anns[0] and not anns[0]['bbox'] == []:
	res.dataset['categories'] = copy.deepcopy(self.dataset['categories'])
	for id, ann in enumerate(anns):
	bb = ann['bbox']
	x1, x2, y1, y2 = [bb[0], bb[0]+bb[2], bb[1], bb[1]+bb[3]]
	if not 'segmentation' in ann:
	ann['segmentation'] = [[x1, y1, x1, y2, x2, y2, x2, y1]]
	ann['area'] = bb[2]*bb[3]
	ann['id'] = id+1
	ann['iscrowd'] = 0
	elif 'segmentation' in anns[0]:
	res.dataset['categories'] = copy.deepcopy(self.dataset['categories'])
	for id, ann in enumerate(anns):
	# now only support compressed RLE format as segmentation results
	ann['area'] = maskUtils.area(ann['segmentation'])
	if not 'bbox' in ann:
	ann['bbox'] = maskUtils.toBbox(ann['segmentation'])
	ann['id'] = id+1
	ann['iscrowd'] = 0
	elif 'keypoints' in anns[0]:
	res.dataset['categories'] = copy.deepcopy(self.dataset['categories'])
	for id, ann in enumerate(anns):
	s = ann['keypoints']
	x = s[0::3]
	y = s[1::3]
	x0,x1,y0,y1 = np.min(x), np.max(x), np.min(y), np.max(y)
	ann['area'] = (x1-x0)*(y1-y0)
	ann['id'] = id + 1
	ann['bbox'] = [x0,y0,x1-x0,y1-y0]

When evaluating across different scales ("small", "medium", and "large"), both ground-truth (g) and prediction (d) will be either ignored or considered according to whether their "area" field is within the specific area range,

cocoapi/PythonAPI/pycocotools/cocoeval.py

Lines 251 to 252 in 8c9bcc3

	if g['ignore'] or (g['area']<aRng[0] or g['area']>aRng[1]):
	g['_ignore'] = 1

cocoapi/PythonAPI/pycocotools/cocoeval.py

Lines 297 to 298 in 8c9bcc3

	# set unmatched detections outside of area range to ignore
	a = np.array([d['area']<aRng[0] or d['area']>aRng[1] for d in dt]).reshape((1, len(dt)))

So, given that the "area" field of ground-truth refers to the number of pixels in the segmentation mask, does this different processing between a ground-truth and a prediction correctly characterize the performance of an object or keypoints detector? As far as I know, there are many object instances of some categories (e.g., "person") of which the value of bbox area and segmentation area are quite different.

[BishwaBS]

@Johnqczhang Do we have updated repo with the updated code that considers the segmentation mask area instead of bbox area while evaluating "segm"?

[kadattack]

expected result: 2765.1486500000005
actual result: 2760

expected result: 1545.4213000000007
actual result: 1542

Does this mean my approach is wrong ? Or is this a consequence of using LUA instead of Python ?
More important, will this cause problems during the training session ?
Thanks
Nico

Has anyone figured out why this is happening with @nico00 results ?
I noticed the same issue with instances_val2017.json
And the same issue is for the BBox when using pycocotools._mask.toBbox() . The numbers seem to be off slightly. Will this cause problems?