Info about datasets

[bhargav-sudo]

Hi,
Why does annotation was not there on the downloaded dataset images ..?
Why the structure of datasets like market1510 , yolo ,pascal VOC etc.. in CVAT looks different when compare to original..?
Can we able to make the structure of datasets similar to the original one..?

Thanks.

[nmanovic]

@bhargav-sudo , thanks for your comments. Could you please describe in more details and give us a couple of examples? What is the difference?

[bhargav-sudo]

Hi,

lets take some of the dataset like Market-1510.
When I downloaded the Market-1510 dataset from the online.
it has a folders of Train , test etc..

I used CVAT tool for re-identification. First I had done detection and the re-identification using auto annotation.
When I downloaded that dataset from CVAT. The folder has only some images(save images from the Export dataset) and txt file.

So, what's the difference here..?
and How can I make CVAT Market-1510 dataset into original one..?

Thanks.

[nmanovic]

@kirill-sizov , could you please look at answer?

[bhargav-sudo]

Hi guys,
any update..?

Thanks.

[sizov-kirill]

Hi guys, any update..?

Thanks.

Hi, do you use Tag=market-1501 during your annotation process in CVAT?

[bhargav-sudo]

Thanks for the reply @kirill-sizov.

Am new to this. where can i find that..?

[sizov-kirill]

If you want to get expected result of exporting into the Market-1501 you should use Label with value market-1501 and attributes: query, person_id, camera_id. Particular use case for Market-1501 dataset it's when you have cropped images with people: one person per image. And after that using CVAT you can fill information about person for each image: query, person_id, camera_id. We implemented such logic based on original Market-1501 dataset.

It will be interesting to get more information about your use case: what kind of images do you have? what types of annotation objects did you try to use in CVAT for getting Market-1501 dataset?

[bhargav-sudo]

Thanks,

I just annotated the data with person detection and next with reidentification.
Then i exported the dataset from the option that you had provided.
I didn't attached any tags.

I want the dataset like you mentioned in above.

[bhargav-sudo]

First thing I want to know how to set these tags that you said in the above message.

Coming to our use case. We need the dataset from CVAT like the original one from the online.

[bhargav-sudo]

HI, Is this is the thing that you are trying to mention.

Let me know if it is wrong or right..?
If right what needs to be update in query attribute..?
if right what needs to do after...?

Thanks,

[sizov-kirill]

It's better to specify type for attributes, try to use this labels definition in Raw (instead of Constructor):

[
  {
    "name": "market-1501",
    "color": "#e2c8c0",
    "attributes": [
      {
        "name": "person_id",
        "input_type": "number",
        "mutable": false,
        "values": [
          "0",
          "1000",
          "1"
        ]
      },
      {
        "name": "camera_id",
        "input_type": "number",
        "mutable": false,
        "values": [
          "0",
          "100", 
          "1"
        ]
      },
      {
        "name": "query",
        "input_type": "checkbox",
        "mutable": false,
        "values": [
          "true"
        ]
      }
    ]
  }
]

Also don't forget to change values for attributes camera_id and person_id according your dataset (convention for list of values: "values"=["start", "stop", "step"])

Once you create a task with a label definition like this, you can annotate your images with tags, see the documentation to read more about this here and about the attribute annotation mode here.

[bhargav-sudo]

Thanks @kirill-sizov.
Will do this and let you know.

Thanks.

[bhargav-sudo]

HI @kirill-sizov
It works.

I had a doubt what does the folder "query" called..?
How it get structured..? Does it selects one random photo from the video or it selects one random photo from one person (like 2 photos for 2 persons).

Does CVAT provides any option to crop the images. ? If not how can you crop them ..?

Thanks.

[bhargav-sudo]

HI @nmanovic. I had tried adding my own dl models into nuclio.
CVAT uses openvino_2020.2 version of models. I modified the code to use other versions of openvino or other type of models in the same version. (like 0031 reidentiication model instead of 0300).
I tried for detection models too. The function has deployed completely.

But it didn't got added to the list of CVAT models.

Any reasons and how to solve it.?
Thanks.

[zhiltsov-max]

I had a doubt what does the folder "query" called..?
How it get structured..? Does it selects one random photo from the video or it selects one random photo from one person (like 2 photos for 2 persons).

You can find a simple explanation of query and gallery subsets in ReID context here and here. Basically, "query" subset means "identities to be classified".

[bhargav-sudo]

Thanks @zhiltsov-max.

So, for every person we have one query image to match with the gallery images.

[bhargav-sudo]

HI @nmanovic. I had tried adding my own dl models into nuclio. CVAT uses openvino_2020.2 version of models. I modified the code to use other versions of openvino or other type of models in the same version. (like 0031 reidentiication model instead of 0300). I tried for detection models too. The function has deployed completely.

But it didn't got added to the list of CVAT models.

Any reasons and how to solve it.? Thanks.

Hi. Any updates..?

Thanks

[bhargav-sudo]

If you want to get expected result of exporting into the Market-1501 you should use Label with value market-1501 and attributes: query, person_id, camera_id. Particular use case for Market-1501 dataset it's when you have cropped images with people: one person per image. And after that using CVAT you can fill information about person for each image: query, person_id, camera_id. We implemented such logic based on original Market-1501 dataset.

It will be interesting to get more information about your use case: what kind of images do you have? what types of annotation objects did you try to use in CVAT for getting Market-1501 dataset?

Hi @kirill-sizov we are still working on it. I just need the dataset of market-1501 from CVAT to be the same with the original one which we can get from online.
we are annotating on a video. (first detection model and reid model). I updated the labels as same as you said before. And I am not able get the folders like query, gt_query, gt_bbox when we export the dataset into market-1501.
But as you mentioned before about cropping images.
(like taking cropped images and attaching the labels or how can i do that..?)

[zhiltsov-max]

But as you mentioned before about cropping images.
(like taking cropped images and attaching the labels or how can i do that..?)

Hi, you could crop images with Datumaro. How would you like to crop them? By fixed coordinates, by a bbox or something else? From the conversation above, I can assume, you would like to auto-annotate images with a model and then crop by a bbox, is it correct?

[bhargav-sudo]

Hi @zhiltsov-max

I want to crop the images using the data of person that i get from the CVAT tool.

Yes, you are correct.
we need the data of person_reid to train the model.
And we need the that data in the market-1501 format where it has query, gt_query, gt_bbox these folder structure.

How can I get them..?

[bsekachev]

Sorry for lack of response from our side. There are too many issues opened. I am trying to reduce them now and I will close this issue.

Please, if the question is still relevant, let us know and do not hesitate to reopen.