Improved interface of interactors on UI #2054

bsekachev · 2020-08-19T09:40:19Z

Motivation and context

How has this been tested?

Manual testing

Checklist

I submit my changes into the develop branch
I have added description of my changes into CHANGELOG file
I have updated the documentation accordingly
I have added tests to cover my changes
I have linked related issues (read github docs)
I have increased versions of npm packages if it is necessary (cvat-canvas,
cvat-core, cvat-data and cvat-ui)

License

I submit my code changes under the same MIT License that covers the project.
Feel free to contact the maintainers if that's a concern.
I have updated the license header for each file (see an example below)

# Copyright (C) 2020 Intel Corporation
#
# SPDX-License-Identifier: MIT

Checklist:

UI notifications & lock when the process is active (UI feedback)
Freeze history when updated points
Add description as a tooltip
Change icon for AI tools
Remove old DEXTR code
CTRL modifier key to maintain requests
Update server interactors API (need some additional parameters and negative points)

coveralls · 2020-08-19T10:02:04Z

Pull Request Test Coverage Report for Build 7214

0 of 0 changed or added relevant lines in 0 files are covered.
140 unchanged lines in 4 files lost coverage.
Overall coverage decreased (-0.07%) to 69.681%

Files with Coverage Reduction	New Missed Lines	%
cvat/apps/engine/media_extractors.py	5	76.11%
src/annotations.js	5	48.63%
src/annotations-history.js	14	30.77%
src/session.js	116	53.97%

Totals
Change from base Build 7202:	-0.07%
Covered Lines:	11753
Relevant Lines:	16412

💛 - Coveralls

cvat-canvas/README.md

cvat-ui/src/utils/typescript.ts

cvat-ui/src/utils/interactors/openvino.dextr.ts

cvat-ui/src/utils/dextr-utils.ts

cvat-ui/src/reducers/models-reducer.ts

bsekachev · 2020-08-20T11:04:07Z

@nmanovic, @ActiveChooN

The PR is ready to review

ActiveChooN

Have a strange issue with canvasModel.ts

Have an issue with a label selection.

Have an issue with interaction.

cvat-canvas/src/typescript/canvasModel.ts

cvat-ui/src/assets/ai-tools.svg

cvat-ui/src/icons.tsx

cvat-canvas/src/typescript/canvasModel.ts

cvat-canvas/src/typescript/crosshair.ts

cvat-canvas/src/typescript/interactionHandler.ts

bsekachev · 2020-08-21T14:55:51Z

Have a strange issue with canvasModel.ts

It had the same behaviour with export Configuration.
Typescript/Babel/Wepback issue.
Nothing critical: babel/babel#8639 . Can try workaround to fix it.

Have an issue with a label selection.

Fixed

Have an issue with interaction.

Hmm. Cannot reproduce. I will try more, but try to restart server, may be it will help.

bsekachev · 2020-08-21T15:07:26Z

Have an issue with interaction.

I suppose it was related with label selector (wrong onChange function as I later found). Should be fixed for now.

ActiveChooN · 2020-08-24T09:57:21Z

cvat-ui/src/components/annotation-page/standard-workspace/controls-side-bar/tools-control.tsx

+            const result = await core.lambda.call(jobInstance.task, interactor, {
+                task: jobInstance.task,
+                frame,
+                points: convertShapesForInteractor((e as CustomEvent).detail.shapes),
+            });


As i understand, negative points in this patch works like positive, isn't it? Maybe it's better add a filter for its or something else, what do you think?

Right now, yes.
Need to update server API for interactors (an interactor on the server should itself decide what to do with negative points)
We can filter these points for DEXTR only on UI (till the next PR), but what problem we are trying to solve?

It's a little bit confusing that negative points works like positive. Maybe it's better to temporary disable ability to create negative points in UI, since we have no interactors to work with and in the next PR with extended interactors functionality enable it.

And in the next PR when we enable them, they will be confusing again.. Or maybe user will input some negative points in the future and they are ignored, what also can be confusing for him, or a user inputs four points (two are accidentaly negative), they are filtered and dextr gets two points instead of 4. Here are a lot of open questions. If it makes you worried, we can discuss it tomorrow on the meeting

cvat-canvas/src/typescript/interactionHandler.ts

nmanovic · 2020-08-26T16:03:25Z

@bsekachev , it is a great feature! It works like a charm. Please give me a day to experiment with it. Thanks!

nmanovic · 2020-08-27T10:12:58Z

@bsekachev ,

When I press ESC, the drawn polygon isn't removed. Is it expected?

nmanovic · 2020-08-27T10:20:32Z

@bsekachev , should we introduce semi-automatic hint in addition to MANUAL and AUTOMATIC? What is your opinion?

nmanovic · 2020-08-27T10:25:44Z

@bsekachev , should we allow to move to another frame during interaction? For now CVAT doesn't and it is good from my point of view. The reason of the question is because for another drawing methods we allow. Should it be unified?

bsekachev · 2020-08-27T15:01:50Z

When I press ESC, the drawn polygon isn't removed. Is it expected?

No, It is expected that drawing will canceled.
I will fix it

bsekachev · 2020-08-27T15:03:48Z

@bsekachev , should we introduce semi-automatic hint in addition to MANUAL and AUTOMATIC? What is your opinion?

If we get a request for that, we can. But right now I would suggest left it as is.

bsekachev · 2020-08-27T15:09:26Z

should we allow to move to another frame during interaction? For now CVAT doesn't and it is good from my point of view. The reason of the question is because for another drawing methods we allow. Should it be unified?

Historically we didn't strict moving between frames during drawing
Because of obvious reasons moving is locked during redrawing an object or interaction
Restricting it in regular drawing might bring inconvenience for users, but I am not sure. We can try and reject it in case of negative feedback

…' into bs/canvas_interaction

nmanovic

LGTM

Draft version

7dc7d50

bsekachev added this to In progress in UI via automation Aug 19, 2020

bsekachev added the enhancement New feature or request label Aug 19, 2020

bsekachev added this to the 1.1.0-release milestone Aug 19, 2020

bsekachev changed the title ~~[WIP] Improved interface of UI Interactors~~ [WIP] Improved interface of interactors on UI Aug 19, 2020

Removed extra file

cc78167

bsekachev commented Aug 19, 2020

View reviewed changes

bsekachev added 9 commits August 19, 2020 17:34

Removed extra code

2cab41f

Updated icon: magic wand

199c264

Ctrl modifier, fixed some cases when interaction event isn't raised

c3132d1

Added tooltip description of an interactor

c9dab60

Locking UI while server fetching

2ad4ae6

Removing old code & refactoring

c9fd42e

Fixed couple of bugs

e477092

Updated CHANGELOG.md, updated versions

6b17687

Merged develop

39cb399

bsekachev requested a review from nmanovic as a code owner August 20, 2020 11:03

bsekachev changed the title ~~[WIP] Improved interface of interactors on UI~~ Improved interface of interactors on UI Aug 20, 2020

bsekachev requested a review from ActiveChooN August 20, 2020 11:04

Update crosshair.ts

61f7104

UI automation moved this from In progress to Review in progress Aug 21, 2020

ActiveChooN suggested changes Aug 21, 2020

View reviewed changes

bsekachev added 2 commits August 21, 2020 17:33

Merge branch 'develop' into bs/canvas_interaction

7a3015c

Minor fixes

057f41a

Fixed eslint issues

e757383

ActiveChooN suggested changes Aug 24, 2020

View reviewed changes

ActiveChooN previously approved these changes Aug 26, 2020

View reviewed changes

Nikita Manovich added 3 commits August 27, 2020 20:21

Initial version of FBRS interactive segmentation

25df34b

Merge remote-tracking branch 'origin/nm/fbrs_interactive_segmentation…

7f81a9d

…' into bs/canvas_interaction

Fix fbrs model_handler

820e498

nmanovic dismissed ActiveChooN’s stale review via 820e498 August 27, 2020 17:40

bsekachev added 2 commits August 28, 2020 12:36

Fixed couple of minor bugs

daeaf75

Added ability to interrupt interaction

9e7ebcc

bsekachev mentioned this pull request Aug 31, 2020

Running detectors on the fly #2102

Merged

11 tasks

nmanovic previously approved these changes Aug 31, 2020

View reviewed changes

UI automation moved this from Review in progress to Reviewer approved Aug 31, 2020

Nikita Manovich added 2 commits August 31, 2020 19:15

Merge remote-tracking branch 'origin/develop' into bs/canvas_interaction

1359c36

Update CHANGELOG.md

ba79862

nmanovic dismissed their stale review via ba79862 August 31, 2020 16:17

UI automation moved this from Reviewer approved to Review in progress Aug 31, 2020

nmanovic previously approved these changes Aug 31, 2020

View reviewed changes

UI automation moved this from Review in progress to Reviewer approved Aug 31, 2020

Delete function.yaml

8ecff7b

nmanovic dismissed their stale review via 8ecff7b August 31, 2020 19:19

UI automation moved this from Reviewer approved to Review in progress Aug 31, 2020

Removed fbrs files

cedf917

nmanovic merged commit 908e056 into develop Aug 31, 2020

UI automation moved this from Review in progress to Done Aug 31, 2020

nmanovic deleted the bs/canvas_interaction branch August 31, 2020 19:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improved interface of interactors on UI #2054

Improved interface of interactors on UI #2054

bsekachev commented Aug 19, 2020 •

edited

coveralls commented Aug 19, 2020 •

edited

bsekachev commented Aug 20, 2020

ActiveChooN left a comment

bsekachev commented Aug 21, 2020

bsekachev commented Aug 21, 2020 •

edited

ActiveChooN Aug 24, 2020

bsekachev Aug 24, 2020 •

edited

ActiveChooN Aug 24, 2020

bsekachev Aug 24, 2020

nmanovic commented Aug 26, 2020 •

edited

nmanovic commented Aug 27, 2020

nmanovic commented Aug 27, 2020

nmanovic commented Aug 27, 2020

bsekachev commented Aug 27, 2020

bsekachev commented Aug 27, 2020

bsekachev commented Aug 27, 2020

nmanovic left a comment

Improved interface of interactors on UI #2054

Improved interface of interactors on UI #2054

Conversation

bsekachev commented Aug 19, 2020 • edited

Motivation and context

How has this been tested?

Checklist

License

coveralls commented Aug 19, 2020 • edited

Pull Request Test Coverage Report for Build 7214

💛 - Coveralls

bsekachev commented Aug 20, 2020

ActiveChooN left a comment

Choose a reason for hiding this comment

bsekachev commented Aug 21, 2020

bsekachev commented Aug 21, 2020 • edited

ActiveChooN Aug 24, 2020

Choose a reason for hiding this comment

bsekachev Aug 24, 2020 • edited

Choose a reason for hiding this comment

ActiveChooN Aug 24, 2020

Choose a reason for hiding this comment

bsekachev Aug 24, 2020

Choose a reason for hiding this comment

nmanovic commented Aug 26, 2020 • edited

nmanovic commented Aug 27, 2020

nmanovic commented Aug 27, 2020

nmanovic commented Aug 27, 2020

bsekachev commented Aug 27, 2020

bsekachev commented Aug 27, 2020

bsekachev commented Aug 27, 2020

nmanovic left a comment

Choose a reason for hiding this comment

bsekachev commented Aug 19, 2020 •

edited

coveralls commented Aug 19, 2020 •

edited

bsekachev commented Aug 21, 2020 •

edited

bsekachev Aug 24, 2020 •

edited

nmanovic commented Aug 26, 2020 •

edited