[NER Fine tuning] content selection

[Amelie-V]

Multi word

Actual state : (VIEW SS)
1- I select various words, highlight is grey and in a solid block (Highlight/words).
2- When selection is done, highlight selection is splited and label selector appears.

• Should be:

1- I select various words, highlight is grey and splited (highlight/word)
2- When selection is done, highlight selection is a solid block label selector appears.

Delete labelling

• Make clicable the whole tooltip to delete

Selection on a searched word

• Selection highlight should not be cut (SS)
• When selection is containing a search word the label selector does not appear (how it works only on right>left sense)
• In general : change appearance of results : in place of Orange highlight show text in bold

Cursor

• Active "hand" cursor (pointer) on piece of text already annotated/Predicted
• Active "Text Select" cursor on the rest of record
• Enlarge the hover state to the whole area : (record + annotated tooltip + empty space between them) (record + predicted tooltip + empty space between us)

New Select label modal

• Integrate new UI modal
• In case of unique label, dont show modal, and just affect label after selecting text
• Add logic to show first and preselected the last label used
• Add following Keyboard shortcut: Enter to valid preselected label, and vertical arrow keyboard or Number to valid other labels

[Amelie-V]

[Amelie-V]

[frascuchon]

Regarding interactions in app, I think it would be nice a way to easily validate a single annotation (entity label in record)

The example below shows a predicted PERSON, where we must navigate over all entities finding predicted one. Even more, by clicking the tooltip, we loose the single annotation, that is a bit confusing.

What do you think @Amelie-V ?

[Amelie-V]

QA 1

General

• "Last label selected" shortcut is always [space]
• [Enter] is the hover state when user is scrolling
• "First label in alphabetic order is [1]
• Continue to number label 9, 10, 11, 12 etc.
• I would recommend to mark numbers label in the legend on top
• Improve hover (see video)
• Extend selector width
• Extend padding around label buttons
• Write shortcut [space];[1] in regular weight (not thin)
• Center sector on highlighted selection

About Predicted text

• On click on text with prediction text should be highlighted in grey

Proposal to speed up annotation on predicted text:

• Activate selector at the same time, and show as dropdown first option the predicted label. Ej: Jerry is predicted as NAME, show NAME as first option.

• Use the "pred. tooltip" to add a validation action. (UI required)

[Amelie-V]

Uploading NER_hover.mov…

[frascuchon]

• Once last label is selected to be presented as the first one, sholdn't be duplicated.

• Introduced highlighting design for text search does not highlight too much. Even more in other tasks:

could we use another highlighting pattern ? maybe filling text with a higher contrast color ?

[dcfidalgo]

@Amelie-V

• To me, it seems a bit confusing that you can remove predictions:

  • a record with predictions in explore mode
  • go to annotation mode, validate only a few predictions in this record
  • go back to explore mode, the predictions are gone

  To me, predictions should be something static that you cannot change from within in web app. Use case: I have predictions from a model, and ask the client to annotate a few records to see how my model is performing. Right now, I cannot visually check this after the client annotated the dataset.

• Like @frascuchon said, I also find it a bit counterintuitive that by clicking the prediction tooltip in the annotate mode, you remove the prediction (or is it the annotation in the end?).

• annotate mode: when clicking on the mention predicted by the model, I would not display the dropdown but automatically choose the label predicted by the model. If you want to change the label, you can simply click again on the mention and choose the right label. As an alternative, I would put the predicted label first in the dropdown.

• right now the workflow works nicely for models that are more or less working, but predictions from a model that makes a lot of mistakes, can get in your way quickly (you first have to remove all the wrong predictions, make your annotations and then press the save button). Then I would rather log the records again without the predictions to the web app. Not sure if two buttons could be a solution:

  • validate: validates the annotations that you explicitly made (by either making new annotations or validating single predictions)
  • validate all: validates all untouched predictions and your explicit annotations

• general comment: when a record is pending, I really miss some "cancel" button next to the "save" button, but my understanding is that right now this is impossible to implement, right @leiyre?

[Amelie-V]

Annotation mode :

• Maintain prediction state when label is annotated
• Add delete all/clean button at record level in all states.
• Add a Pre-annotation state in transparency by default.
• Don't allow delete prediction.
• Review Highlight

[Amelie-V]

Conclusion post Friday review:

Next release MVP:

• Maintain prediction state (don't delete them)
• Pre-annotation (can be deleted on clic)
• Results searched: bold + color 4C4EA3
• Dropdown: Add space between the label sugerency and the full list
• Dropdown: color border is the darker label color
• Maintain numeric until 9
• DSont show num on legend (for now)
• Label sugerency: when there is a prediction: the prediction, when no prediction the last used

V2:

• Add Clear annotations bt
• Add dropdown config.
• Integrate new header (cross tasks)
• prevent shortcut pagination when dropdown is active

[Amelie-V]

• Background dropdown : use hover grey (in place of BG app color)
• Lighter the color border
• I would add the label number on the legend right now
• scroll with arrow doesn't work Versión 98.0.4758.80 (Build oficial) (x86_64)
• Legend : change "show all" for "+3"

[Amelie-V]

[frascuchon]

The "Cancel" button action, but we can tackle in v2.

[Amelie-V]

QA OK