[BUG] Already existing labels not visible in CVAT after annotation request

[tvoirand]

Describe the problem

I created a FiftyOne dataset including already existing labels from files on my disk.
After sending an annotation request to CVAT, I expected to see the already existing labels in CVAT, but they are not visible there.

I am probably missing some step.
I searched for similar issues, and only found this one. But the label field, which was the cause of this other issue, is correct in my case.

Code to reproduce issue

The code I am using to request the annotation:

dataset = fo.load_dataset("deers_dataset")
dataset.annotate(
	annotation_key,
	label_field="ground_truth",
	label_type="detections",
	project_id=project_id,
	task_name=annotation_key,
	username=config["cvat"]["cvat_user"],
	password=config["cvat"]["cvat_password"],
	url=f"http://{config['cvat']['cvat_host']}:{config['cvat']['cvat_port']}",  # noqa: E231
)

Here is the output of print(dataset):

Name:        deers_dataset
Media type:  image
Num samples: 5
Persistent:  True
Tags:        []
Sample fields:
    id:           fiftyone.core.fields.ObjectIdField
    filepath:     fiftyone.core.fields.StringField
    tags:         fiftyone.core.fields.ListField(fiftyone.core.fields.StringField)
    metadata:     fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.metadata.ImageMetadata)
    ground_truth: fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.labels.Detections)

And a sample from print(dataset.head()):

<Sample: {
    'id': '6596bcdbf1f603988cd7d73f',
    'media_type': 'image',
    'filepath': '/data3/tvoirand/fiftyone/data/test_data/20231115_deers/stockvault-deer143264.jpg',
    'tags': [],
    'metadata': <ImageMetadata: {
        'size_bytes': 174954,
        'mime_type': 'image/jpeg',
        'width': 1000,
        'height': 516,
        'num_channels': 3,
    }>,
    'ground_truth': <Detections: {
        'detections': [
            <Detection: {
                'id': '6596bcdbf1f603988cd7d73b',
                'attributes': {},
                'tags': [],
                'label': 'Vegetal',
                'bounding_box': [0.1, 0.3875968992248062, 0.1, 0.1937984496124031],
                'mask': array([[1, 1, 1, ..., 1, 1, 1],
                       [1, 1, 1, ..., 1, 1, 1],
                       [1, 1, 1, ..., 1, 1, 1],
                       ...,
                       [1, 1, 1, ..., 1, 1, 1],
                       [1, 1, 1, ..., 1, 1, 1],
                       [1, 1, 1, ..., 1, 1, 1]], dtype=uint8),
                'confidence': None,
                'index': None,
                'check': 'add',
                'uuid': 'f090bc96-a6c2-4987-87dd-90a3a95ef682',
            }>,
        ],
        'tags': ['stockvault-deer143264_qr'],
        'geojson_path': '/data3/tvoirand/fiftyone/data/test_data/20231115_deers/stockvault-deer143264_qr.geojson',
    }>,
}>

System information

• OS Platform and Distribution: Ubuntu 22.04.3 LTS
• Python version: Python 3.12.0
• FiftyOne version: 0.23.1
• FiftyOne installed from: pip

Willingness to contribute

The FiftyOne Community encourages bug fix contributions. Would you or another
member of your organization be willing to contribute a fix for this bug to the
FiftyOne codebase?

• Yes. I can contribute a fix for this bug independently
• Yes. I would be willing to contribute a fix for this bug with guidance
  from the FiftyOne community
• No. I cannot contribute a bug fix at this time

[tvoirand]

Actually in my case the label_field was correct but the label_type was incorrect.

The mask in my labels were being dropped by annotate(), because I had set label_type="detections". My labels are correctly forwarded to CVAT when using label_type="instances".

This is because instance segmentations (fo.Detection with mask) is treated differently from bounding box detections (fo.Detection without mask) since these are two different annotation tools in CVAT.

I got support on the FiftyOne Slack to solve this, but it was actually documented in the docs (see label_type in the annotation tasks parameters description).