You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Once the annotator completes his job he should be able to click a button that says "Commit" or "Job Complete" to let the annotation approver know that the annotator has completed his job and it's time for the annotation approver to approve the annotations.
Similarly, we may want to have a similar button for annotation-approver to commit after approving certain annotations.
The implementation for what needs to happen after the commit button is clicked in each of the above cases could be defined as an interface such as def perform_action_on_annotator_commit: def perform_action_on_annotation_approver_commit:
which basically can be implemented by anyone who adopts this project.
Advantages of having the Job Complete or Commit button:
Better user experience for the annotator/approver mentioning that they completed the job!
The approver need not keep looking at the UI or does not need to communicate with the annotator about if the annotator has completed his Job.
Having this feature will give greater flexibility to companies looking to adopt an open-source tool such as Doccano and will give Doccano the edge to get adopted more widely and integrated with their workflow.
Disadvantages:
We will need to work through the workflow and decide if the annotator can Commit or Complete Job multiple times, if yes how does the annotation approver know that this is the final commit by the annotator? Perhaps we could do auto-commits for the annotator's role after say every X (500/configurable) utterances are annotated to continuously get feedback from the annotation approver.
Example:
implementation for perform_action_on_annotator_commit: would be to send email to a list of annotation approvers on the project.
implementation for perform_action_on_annotation_approver_commit: would be to send email to the project owner and also trigger a job after label completion automatically.
Open for discussion and ideas on how we can better implement this!!
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
User experience/workflow improvement
Once the annotator completes his job he should be able to click a button that says "Commit" or "Job Complete" to let the annotation approver know that the annotator has completed his job and it's time for the annotation approver to approve the annotations.
Similarly, we may want to have a similar button for annotation-approver to commit after approving certain annotations.
The implementation for what needs to happen after the commit button is clicked in each of the above cases could be defined as an interface such as
def perform_action_on_annotator_commit:
def perform_action_on_annotation_approver_commit:
which basically can be implemented by anyone who adopts this project.
Advantages of having the
Job Complete
orCommit
button:Disadvantages:
We will need to work through the workflow and decide if the annotator can
Commit
orComplete Job
multiple times, if yes how does the annotation approver know that this is the final commit by the annotator? Perhaps we could do auto-commits for the annotator's role after say every X (500/configurable) utterances are annotated to continuously get feedback from the annotation approver.Example:
perform_action_on_annotator_commit:
would be to send email to a list of annotation approvers on the project.perform_action_on_annotation_approver_commit:
would be to send email to the project owner and also trigger a job after label completion automatically.Open for discussion and ideas on how we can better implement this!!
Beta Was this translation helpful? Give feedback.
All reactions