Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support relative paths in import and export #1463

Merged
merged 125 commits into from Jun 19, 2020
Merged

Support relative paths in import and export #1463

merged 125 commits into from Jun 19, 2020

Conversation

zhiltsov-max
Copy link
Contributor

@zhiltsov-max zhiltsov-max commented Apr 24, 2020 •
edited

Depends on #1715

Motivation and context

Added support for importing annotations with relative paths.
Fixed file name matching when importing annotations with relative paths and the target task has been created with images from file share.

How to test:

  • create a task with relative paths
  • import and export annotations. Images with same names lying in different directories should be handled normally.

How has this been tested?

Unit tests

Checklist

  • I submit my changes into the develop branch
  • I have added description of my changes into CHANGELOG file
  • I have updated the documentation accordingly
  • I have added tests to cover my changes
  • I have linked related issues (read github docs)

License

  • I submit my code changes under the same MIT License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below)
# Copyright (C) 2020 Intel Corporation
#
# SPDX-License-Identifier: MIT

@zhiltsov-max
Move annotations to dm
9c87cdc
@zhiltsov-max
Refactor dm
06d6217
@zhiltsov-max
Rename data manager
40375f0
@zhiltsov-max
Move anno dump and upload functions
8c64a9a
@zhiltsov-max
Join server host and port in cvat cli
ce9c0a8
@zhiltsov-max
Move export templates dir
cf181ae
@zhiltsov-max
add dm project exporter
76c48c7
@zhiltsov-max
update mask format support
a2f00f5
@zhiltsov-max
Use decorators for formats definition
37a4a03
@zhiltsov-max
Update formats
7736039
@zhiltsov-max
Update format implementations
ba75d7c
@zhiltsov-max
remove parameter
8b0173a
@zhiltsov-max
Add dm views
5c770b4
@zhiltsov-max
Move annotation components to dm
527123b
@zhiltsov-max
restore extension for export formats
a758550
@zhiltsov-max
update rest api
81404b1
@zhiltsov-max
use serializers, update views
92b1bba
@zhiltsov-max
merge develop
ae7974e
@zhiltsov-max
Update format names
f8e2663
@zhiltsov-max
Update docs
927ca56
@zhiltsov-max
Update tests
cc04df1
@zhiltsov-max
move test
2d2ca2d
@zhiltsov-max
fix import
0263a0e
@zhiltsov-max
Extend format tests
ec52ecd
@zhiltsov-max
django compatibility for directory access
76f496e
@zhiltsov-max
move tests
1f4612c
@zhiltsov-max
update module links
e139e98
@zhiltsov-max
fixes
acf79ca
@zhiltsov-max
fix git application
c3d8ed1
@zhiltsov-max
fixes
5567e54
@azhavoro
Copy link
Contributor

@zhiltsov-max could you please fix conflicts?

This was referenced Jun 15, 2020
Merged
Merged
@zhiltsov-max zhiltsov-max changed the title [WIP] [Datumaro] Relative image paths support [WIP] Support relative paths in import and export Jun 16, 2020
@zhiltsov-max
Copy link
Contributor Author

@azhavoro,

Problem 1 - should be resolved by #1615 and #1728
Problem 2 - resolved in #1638

@zhiltsov-max zhiltsov-max changed the title [WIP] Support relative paths in import and export [Dependent] Support relative paths in import and export Jun 16, 2020
@cvat-ai cvat-ai deleted a comment from zhiltsov-max Jun 17, 2020
@zhiltsov-max zhiltsov-max changed the title [Dependent] Support relative paths in import and export Support relative paths in import and export Jun 17, 2020
@nmanovic
Copy link
Contributor

@zhiltsov-max , could you please resolve conflict?
@azhavoro , could you please test the patch ASAP? Need to merge it till end of week.

@nmanovic Codacy Production
Copy link
Contributor

Codacy Here is an overview of what got changed by this pull request:

Issues
======
+ Solved 1
           

Complexity increasing per file
==============================
- cvat/apps/dataset_manager/formats/yolo.py  2
- cvat/apps/dataset_manager/bindings.py  1

See the complete overview on Codacy

@azhavoro
Copy link
Contributor

azhavoro commented Jun 18, 2020 •
edited

@azhavoro,

Problem 1 - should be resolved by #1615 and #1728
Problem 2 - resolved in #1638

@zhiltsov-max uploading of MOT still doesn't work for me.

@zhiltsov-max
Copy link
Contributor Author

@azhavoro, could you share the MOT annotations?

@azhavoro
Copy link
Contributor

@azhavoro, could you share the MOT annotations?

task_task-2020_06_19_09_15_02-mot 1.1.zip
task_task-2020_06_19_09_15_02-cvat for images 1.1.zip

@zhiltsov-max zhiltsov-max mentioned this pull request Jun 19, 2020
Merged
8 tasks
Dataset framework (Datumaro) automation moved this from Review in progress to Reviewer approved Jun 19, 2020
nmanovic
nmanovic approved these changes Jun 19, 2020
View reviewed changes
Copy link
Contributor

@nmanovic nmanovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nmanovic nmanovic merged commit 0eb005c into develop Jun 19, 2020
Dataset framework (Datumaro) automation moved this from Reviewer approved to Done Jun 19, 2020
@bsekachev bsekachev deleted the zm/dm-relative-image-paths branch June 25, 2020 10:50
frndmg pushed a commit to signatrix/cvat that referenced this pull request Aug 5, 2020
@zhiltsov-max @nmanovic
Support relative paths in import and export (cvat-ai#1463)
3245496 
* Move annotations to dm

* Refactor dm

* Rename data manager

* Move anno dump and upload functions

* Join server host and port in cvat cli

* Move export templates dir

* add dm project exporter

* update mask format support

* Use decorators for formats definition

* Update formats

* Update format implementations

* remove parameter

* Add dm views

* Move annotation components to dm

* restore extension for export formats

* update rest api

* use serializers, update views

* merge develop

* Update format names

* Update docs

* Update tests

* move test

* fix import

* Extend format tests

* django compatibility for directory access

* move tests

* update module links

* fixes

* fix git application

* fixes

* add extension recommentation

* fixes

* api

* join api methods

* Add trim whitespace to workspace config

* update tests

* fixes

* Update format docs

* join format queries

* fixes

* update new ui

* ui tests

* old ui

* update js bundles

* linter fixes

* add image with loader tests

* fix linter

* fix frame step and frame access

* use server file name for annotations export

* update cvat core

* add import hack for rest api tests

* move cli tests

* fix cvat format converter args parsing

* remove folder on extract error

* print error message on incorrect xpath expression

* use own categories when no others exist

* update changelog

* really add text to changelog

* Fix annotation window menu

* fix ui

* fix replace

* update extra apps

* format readme

* readme

* linter

* Fix old ui

* Update CHANGELOG.md

* update user guide

* linter

* more linter fixes

* update changelog

* Add image attributes

* add directory check in save image

* update image tests

* update image dir format with relative paths

* update datumaro format

* update coco format

* update cvat format

* update labelme format

* update mot format

* update image dir format

* update voc format

* update mot format

* update yolo format

* update labelme test

* update voc format

* update tfrecord format

* fixes

* update save_image usage

* remove item name conversion

* fix merge

* fix export

* prohibit relative paths in labelme format

* Add test for relative name matching

* move code

* implement frame matching

* fix yolo

* fix merge

* fix merge

* prettify code

* fix methid call

* fix frame matching in yolo

* add tests

* regularize function output

* update changelog

* fixes

* fix z_order use

* fix slash replacement

* linter

* t

* t2

Co-authored-by: Nikita Manovich <40690625+nmanovic@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nmanovic nmanovic nmanovic approved these changes

@azhavoro azhavoro azhavoro approved these changes

@bsekachev bsekachev Awaiting requested review from bsekachev

Assignees
No one assigned
Labels
None yet
Projects
Dataset framework (Datumaro)
  
Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@zhiltsov-max @coveralls @nmanovic @azhavoro