Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Project export with 3d tasks #3502

Merged
merged 80 commits into from Aug 17, 2021
Merged

Project export with 3d tasks #3502

merged 80 commits into from Aug 17, 2021

Conversation

ActiveChooN
Copy link
Contributor

@ActiveChooN ActiveChooN commented Aug 3, 2021

Motivation and context

Added ability to export project dataset with 3d tasks
Depends on #3365

How has this been tested?

Manually

Checklist

License

  • I submit my code changes under the same MIT License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below)
# Copyright (C) 2021 Intel Corporation
#
# SPDX-License-Identifier: MIT

@@ -940,9 +924,11 @@ def map_label(name): return label_cat.find(name)[0]
return convert_cvat_anno_to_dm(cvat_frame_anno, label_attrs, map_label, self._format_type, self._dimension)

class CVATProjectDataExtractor(datumaro.Extractor, CVATDataExtractorMixin):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think of implementing this class as (roughly):

task_datasets = { task_id: make_task_dataset(task) for task_id, task in tasks }
# append name mangling and other transforms like
for task_id, task_dataset in task_datasets.items():
  task_dataset.transform('rename', ...)

dataset = Dataset.from_extractors(*task_datasets)

@bsekachev bsekachev self-assigned this Aug 11, 2021
@bsekachev
Copy link
Member

I am not expert in cvat/apps/dataset_manager, so I only tried this patch and was able to export both tasks and projects in 3D format. Just one question, if we dump 3D tasks we can only choice between PCD and kitty formats. But when we dump a project with one 3D tasks it shows all the formats (at the same time I cannot create a project with both 3D and 2D data). Is this expected behaviour?

bsekachev
bsekachev previously approved these changes Aug 12, 2021
@bsekachev
Copy link
Member

@zhiltsov-max Do you have any new comments on the PR? I didn't see your approve

@ActiveChooN
Copy link
Contributor Author

But when we dump a project with one 3D tasks it shows all the formats (at the same time I cannot create a project with both 3D and 2D data). Is this expected behaviour?

@bsekachev, fixed

bsekachev
bsekachev previously approved these changes Aug 16, 2021
attributes["track_id"] = -1
index += 1

dm_item = datumaro.DatasetItem(id=osp.split(frame_data.name)[-1].split('.')[0],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Fix to my reported problem 3: Use either os.path.splitext() or str.rsplit(s, maxsplit=1)

  2. Use of split() here does not conform to 2d formats and established CVAT behavior here, because 2d formats preserve the relative path information (to handle the problem with frames with same names in different dirs). Yes, it was discussed in the original 3d PR, but they refused to change it.

@zhiltsov-max
Copy link
Contributor

zhiltsov-max commented Aug 16, 2021

My comments:

  1. image
    image

I think, it would be more user-friendly, if delete had red background and was positioned after exporting. How about renaming "export task dataset" to "export (as) dataset"? Two very similar names confuse.
2. image
After the button is pressed and the menu disappears, when I reopen the menu, it has the button highlighted.
3. Export in sly pcd of a task with images:

и.м.г@ 1.JPEG
им. 536.45.--+=.г 3.jpeg
им.png.г 2.jpg
имг 3 (копия 1).jpeg

Error: Request failed with status code 500. "datumaro.components.errors.RepeatedItemError: Item ('им', 'qqqaaq') is repeated in the source sequence.\n".

  1. 2d project dataset export menu has 3d formats. Task export doesn't
  2. image
    The display of subset names looks a bit unclear to me. Maybe, it should be improved.
  3. I didn't see 2d / 3d task / project indicator anywhere. Maybe, it's worth adding.

@ActiveChooN
Copy link
Contributor Author

@zhiltsov-max

I think, it would be more user-friendly, if delete had a red background and was positioned after exporting.

Fixed. I'm not sure about the red color on menu items, there is already a red button on the delete modal.

Export in sly pcd of a task with images:

Fixed

Other comments are not very related to this PR so I moved them to a separate issue.

@nmanovic nmanovic merged commit cef42b6 into develop Aug 17, 2021
@nmanovic nmanovic deleted the dk/project-export-3d branch August 17, 2021 09:05
@ActiveChooN
Copy link
Contributor Author

@dvkruchinin, hi, can you add tests for this feature, please?

@dvkruchinin
Copy link
Contributor

@dvkruchinin, hi, can you add tests for this feature, please?

Sure. A test will be prepared.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants