Annotation Guideline

[dbrembilla]

Resolves #235
(to be merged only when the feature is complete)

• Adds Annotation Strategy Guideline to the available tabs in the project view.
• Annotation Strategy Guideline can be edited by admins and viewed by editors.
• Annotation Strategy Guidelines can have labels associated. A description, a preferred shape and a reference image can be added
• Users can choose a set of allowed shapes (independent of a label) for enforced guidelines. Only these tools will be available/allowed for annotation.
• Annotation Strategy Guidelines can be enforced for the annotation process, making it so that only labels within a strategy can be used and the preferred shapes are enforced (validated in the backend)
• Enforcement applies to Largo as well, making it so that only labels with a Strategy Guideline can be used and the preferred shapes are enforced (i.e. a label with shape point cannot be changed to a label with shape circle)
• Enforcement applies to LabelBOT as well, so it only suggests "allowed" labels (for the current shape)
• When annotating, description and reference image are displayed

[mzur]

@dbrembilla I will implement the "enforce" switch and the "users can choose a set of allowed shapes (independent of a label)" feature in the show/edit annotation guideline view. After that you can continue to actually implement the guideline in the annotation tools and Largo.

[mzur]

Thinking about this, "enforcing" a guideline has so many implications. Maybe we should discuss these in the group meeting again:

• Is the set of shapes is limited (independent of labels), can the admin still choose one of these for a single label in the guideline? Example: Only circles and polygons are allowed, label A is limited to circles, label B to polygons and label C is not configured so it can be annotated either with circles or polygons.

• Users cannot attach a label to an annotation that has an "incorrect" shape (using the attach or swap tools).

  • We need a helpful error message for this. Users should create a new annotation instead (and maybe delete the old one).

• Largo has to check the shape of each annotation to decide if the label may be attached (or swapped). If anything is wrong, the whole Largo session is not saved.

  • Ignore and not save these cases? Similar to changing the label of other users? Maybe add a new feedback message that this happened. If some selections were not changed, these should still be selected in Largo. Add a "preflight" parameter to check a request without applying it.

• LabelBOT can only suggest labels that have the shape of the (already drawn) annotation. So by choosing the "wrong" shape, users can limit LabelBOT in a way that it can't suggest the correct label any more.

  • LabelBOT should suggest labels as usual but the UI does not allow them to choose labels with "incorrect" shape (or if the labels are not allowed by the guideline in the first place).

[mzur]

@dbrembilla I finished the UI to create/edit annotation guidelines. Going forward I suggest that you implement enforcement of the guidelines (for the annotation tools and Largo) in the backend and I do the frontend. We can talk this through on Monday.

I also thought if "annotation instructions" would fit even better because when I explained this feature in the manual article I was always talking about "instructions".

---

Thoughts about the backend implementation:

• The API endpoints to create image/video annotations (also the bulk endpoint) have to check if there is an enforced guideline. If the volume belongs to many projects and one of the projects has an enforced guideline, the request has to provide the guideline ID that should be applied. If one project has an enforced guideline and the other has none, the request can provide null to say that no guideline should be applied. If all projects have enforced guidelines, one of these has to be chosen.
• The API endpoints to update annotation labels also have to apply a guideline similar to above.
• Largo should validate the annotation labels and shapes based on the (chosen, see above) guideline. If a label is not allowed or an annotation shape does not match the chosen label, this change should be ignored similar to how currently changes are ignored that would delete labels of other users (unless "force" is enabled). Largo should collect all ignored changes (also the ignored changes because of a different user) and return these to the UI, so these can be displayed there along with a warning/info message.