CVAT AI Tools: dextr usage

[aschernov]

My actions before raising this issue

• Read/searched the docs
• Searched past issues

Expected Behaviour

Current Behaviour

1. Annotation speed (manually annotation VS semi-automatic annotation using AI tools: dextr interactor
We made an experiment to compare manually polygon annotation speed and semi-automatic using the dextr interactor and got the following results:

Annotation type	Total polygons	Time	Speed
manually	1335	11,2	119,2
dextr	1450	19,83	73,12

*dextr speed retard is 38,65%

2. DEXTR advantages and disadvantages
Advantages:

• Accuracy regardless of zoom. It isn't necessary to use zoom to get a good annotation result:
  (zoom)
  (without zoom)

• Works well on round-shaped objects located on a contrast background like these ones:

it took 5-10 seconds (put 4 points and wait a request to server) to annotate such objects using DEXTR

• Works well enough on irregularly-shaped objects like trees, vegetation, bushes:
  (7 sec, some corrections were made on the left side)
  (9 sec, but 5 points)
  (This one took 20 sec, but the result is impressive)
  (10 sec, but 5 points)
  (10 sec, but 5 points)

Disadvantages:

• The user interface blocks every time during a server request. A user has to wait for 3-4 seconds with no opportunity to make something useful for annotation

• Too many polygons points, especially for big objects. It affects CVAT performance and complicates corrections:

• Often a completed polygon doesn't include a covered by initial points area:

• Low annotation quality for long, thin, tall and strict-shaped objects:

To get this result with dextr we spent ~40 sec:

And ~ 12 sec to get the same result with manually annotation:

Possible Solution

3. Suggestions how to improve the tool

• Add an opportunity to set a number of initial points. At the moment it's 4 by default but according to a user's experience he can make a conclusion that some objects annotate well enough if he will put, for example, 6 initial points and start the dextr extractor
• Add a visual notification for users that they can press "ctrl" to enlarge number of initial points before and after creating a polygon by dextr
• Reduce a number of points for polygons created by dextr
• Make a request to server invisible for a user and avoid blocking user interface
• Develop a multi-annotation mode for similar objects. For example, we see identical objects on a frame but they have different size. The idea is we could set initial points for these objects and start the dextr after that for all these objects at once.