Wrong Annotation Per Category using COCO Json

[JulienFleuret]

Hello

I finish to annotate a tiny dataset.
This dataset has 4 classes.
I dump the annotations as a MS-COCO json format.

Using pycocotools I ask to access the annotations related to 1 of the 4 classes.
What I got does not correspond to what I ask, In the annotations returned I got annotations that does cover other annotations as well as the one I asked.

I used pycocotools before with different dataset without issues.

----Edit----
I pushed further my investigations... the whole convertion is wrong.
I labeled 200 images only, the number of images id is right but the only image that as been converted is the first one and that for each classe.

[nmanovic]

@JSharp4273 , sorry for the bad experience. Please let's look at the issue and come back with a fix. Could use please provide steps to reproduce the bug? Thus it will be easy for us to fix it in short period of time.

[JulienFleuret]

Actually I just labels my dataset, dump the data as MS-COCO JSON.
Then I wanted to save the mask from the annotations in a folder in order to evaluate if I miss labeled an image.
My first step was to check if the labels names was correct.
There was.
I check that le number of image_ids fit with the number of images I labeled.
Then I wrote the folowing python script:

for cat in  tmp:
    print(len(tmp[cat]))
    anns = tmp[cat]
    for i in range(len(tmp[cat])):
        mask = coco.annToMask(anns[i]) * 255
        output_filename = os.path.join('/home/smile/prog/tiny_labeling_cows/a65/masks', cat,'{}.png'.format(i) )
        cv2.imwrite(output_filename, mask)

coco in this script is an pycocotools.coco.COCO object.

I can provide you the generated json, and cvat xml files if you want to compare.

The cvat xml file seems to be accurate.

[azhavoro]

I can provide you the generated json, and cvat xml files if you want to compare.

Hi @JSharp4273 It would be very helpful.

[JulienFleuret]

Is there any email address I can send it to you ?

[BerryRB]

@azhavoro Hi, when I dump the annotations as a MS-COCO json format, I also have some problems.
I try to do some key annotation on my dataset, the annotations as a MS-COCO json format are as follows:
{
"licenses": [
{
"name": "",
"id": 0,
"url": ""
}
],
"info": {
"contributor": "",
"date_created": "2019-11-06",
"description": "test2",
"url": "",
"version": 6,
"year": "2019"
},
"categories": [
{
"id": 1,
"name": "1",
"supercategory": ""
},
{
"id": 2,
"name": "2",
"supercategory": ""
},
{
"id": 3,
"name": "3",
"supercategory": ""
},
{
"id": 4,
"name": "4",
"supercategory": ""
},
{
"id": 5,
"name": "5",
"supercategory": ""
}
],
"images": [
{
"coco_url": "",
"date_captured": "",
"flickr_url": "",
"license": 0,
"id": 0,
"file_name": "a0aaf5fc88ac2d00306ef5d418c15d34c.jpg",
"height": 224,
"width": 208
}
],
"annotations": []
}
Nothing in "annotations":[]. And the annotations as a CVAT XML for IMAGES is normal, just as follows:
<annotations> <version>1.1</version> <meta> <task> <id>2</id> <name>test2</name> <size>40</size> <mode>annotation</mode> <overlap>0</overlap> <bugtracker/> <created>2019-11-06 06:14:08.415613+00:00</created> <updated>2019-11-06 06:19:33.227751+00:00</updated> <start_frame>0</start_frame> <stop_frame>0</stop_frame> <frame_filter/> <z_order>False</z_order> <labels> <label> <name>1</name> <attributes> </attributes> </label> <label> <name>2</name> <attributes> </attributes> </label> <label> <name>3</name> <attributes> </attributes> </label> <label> <name>4</name> <attributes> </attributes> </label> <label> <name>5</name> <attributes> </attributes> </label> </labels> <segments> <segment> <id>2</id> <start>0</start> <stop>39</stop> <url>http://192.168.40.27:8080/?id=2</url> </segment> </segments> <owner> <username>liurui</username> <email>609213779@qq.com</email> </owner> <assignee/> </task> <dumped>2019-11-06 06:20:11.186720+00:00</dumped> </meta> <image id="0" name="a0aaf5fc88ac2d00306ef5d418c15d34c.jpg" width="208" height="224"> <points label="1" occluded="0" points="49.05,142.47;62.72,95.31"> </points> <points label="2" occluded="0" points="150.44,130.21;121.67,153.31"> </points> <points label="3" occluded="0" points="141.95,76.92;118.84,87.29"> </points> <points label="4" occluded="1" points="189.60,144.40;173.07,165.10"> </points> <points label="5" occluded="1" points="52.35,27.40;37.26,58.53"> </points> <points label="5" occluded="1" points="149.83,30.14;137.72,37.34"> </points> </image> </annotations>
In this annotations, I can get the coordinates of the marked points.

[azhavoro]

@JSharp4273 Could you please attach files to the comment directly or share via any preferable service (dropbox, etc)?

[azhavoro]

@BerryRB Hi, it's expected. For now CVAT supports export of polygons and rectangles only for COCO (https://github.com/opencv/cvat/blob/develop/cvat/apps/annotation/README.md#ms-coco-object-detection).

[BerryRB]

@azhavoro Thank you for your reply. How can I get the coco file about point annotation?

[JulienFleuret]

Here are a zip folder with the zip annotations (cvat xml, coco json)
annotations.zip

[azhavoro]

@BerryRB If you have to use COCO format you can extend existing dump functionality (https://github.com/opencv/cvat/blob/develop/cvat/apps/annotation/coco.py) to handle your case properly. It would be great if you contribute this feature to CVAT.

[azhavoro]

@JSharp4273 Thanks, I'll take a look.

[JulienFleuret]

@azhavoro FYI the annotations for class named animal_poly must overlap (or almost_overlap in some images) with the classes lie_down and stand_up.
If i may help I can also provide you the images

[BerryRB]

@azhavoro Hi, I tried to add the human key annotation function for cvat. Now I have basically achieved this function by modifying the file (https://github.com/opencv/cvat/blob/develop/utils/coco/converter.py),
I want to add this part of modification to source (https://github.com/opencv/cvat/blob/develop/cvat/apps/annotation/coco.py). But I don't know where to call the (https://github.com/opencv/cvat/blob/develop/cvat/apps/annotation/coco.py) code file (in fact, I don't know how to define the "file" object, annotations "in line 61 of (https://github.com/opencv/cvat/blob/develop/cvat/apps/annotation/coco.py)), so I can't debug. Can you help me?

[azhavoro]

@BerryRB Hi, dumper code is called from here https://github.com/opencv/cvat/blob/3b6961f4db95894229f466cd9b897c5ef3f8db9b/cvat/apps/engine/annotation.py#L722

I suggest you make a copy of dumper function and call it from mentioned function directly for debugging, because it's difficult to debug python exec call.

[BerryRB]

@azhavoro Thanks for your repIy. I want to copy dumper function from (https://github.com/opencv/cvat/blob/develop/cvat/apps/annotation/coco.py) for debugging. How should I get the file in the form of (file_object, annotations) in the dump function?

[azhavoro]

@BerryRB I'm not sure that understand you question correctly, but file_object is dump_file defined in the 731th line https://github.com/opencv/cvat/blob/3b6961f4db95894229f466cd9b897c5ef3f8db9b/cvat/apps/engine/annotation.py#L731

I try to explain my previous answer. You code should be smth like that:

# define new function
def coco_dumper_copy(file_object, annotations):
  # copy the dumper code from https://github.com/opencv/cvat/blob/develop/cvat/apps/annotation/coco.py
  # Note also you need copy mask_to_polygon function as well


def dump(self, filename, dumper, scheme, host): 
    # you need to change only the last line:
#   execute_python_code("{}(file_object, annotations)".format(dumper.handler), global_vars)
    coco_dumper_debug(dump_file, anno_exporter)
   

[BerryRB]

@azhavoro So I should "with open(filename, 'wb') as dump_file:" to get dump_file. But I don't know what kind of file filename here refers to. Can you provide me a sample file in filename format？ Thank you

[azhavoro]

@BerryRB Maybe it will be helpful:
Please read the corresponding section of contributing guide https://github.com/opencv/cvat/blob/develop/CONTRIBUTING.md#development-environment
After that you can run CVAT without Docker and easily debug backend python code via Visual Studio Code.

[JulienFleuret]

@azhavoro Have you found why the convertion from CVAT xml to MS-COCO json does not compute more than the first annotated image ?

[azhavoro]

@JSharp4273 Hi, apologies for the delayed response. What version of CVAT did you use? I tested on the current develop branch: I created the task, upload *.xml provided provided by you and dumped annotation as COCO json. It's completely different from your *.json file. Also I tried to upload the generated json and it seems to look ok. Could you please try the latest develop branch?
coco.zip

[JulienFleuret]

@azhavoro I think you confused my name with the one of someone else.
I take a look of the version that I have here I did not find any reference of any version.
How can I know which version I am using ?

[azhavoro]

@JSharp4273 Yep, sorry about that. If you cloned CVAT via git it's possible obtain exact commit that you use by git rev-parse HEAD command. However if you downloaded CVAT as zip you can open link <URL>:<port>/api/v1/server/about. In any case please update to the latest version and try to reproduce the bug, because I cannot reproduce it on my side.

[JulienFleuret]

I took a look when the folder for the version of CVAT I used was created.
I remembered that this folder was created only for this version and never updated.
It was created in august.
I did a git pull to updated, and I rebuild CVAT after.
Do you think it is enough to fix that issue ?

[azhavoro]

@JSharp4273 On my side the latest develop branch works as expected. Since August there have been several fixes to the coco dumper/loader, just try to update. If the update does not solve the issue, let's investigate it further.

[JulienFleuret]

@azhavoro Ok ok I did it I will make a trial tomorrow.

[JulienFleuret]

Hello
I made the trial I am talking about in my last post.
I observed that when I dump the annotation as MS-COCO JSON the annotations fit with the image.
The issue I reported in here does not exist anylonger.
I observed another error but minor, I labeled for this trial a set of 200 images but only have 199 image ids in the MS-COCO JSON.
However this is another error.
Thank you very much for your help.