Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancement/Approve labels #258

Merged
merged 2 commits into from Jun 28, 2019

Conversation

c-w
Copy link
Member

@c-w c-w commented Jun 25, 2019

As discussed in #249, it is a very useful feature to enable a type of users to review and approve labels that other users annotated. The review feature for example enables workflows like the ones discussed in #65 where many users can annotate the dataset and then some trusted users can verify the labels before we use them for downstream workflows such as model building.

As such, this pull request implements a simple label review mechanism: admin users now see a button on the annotation page that enables them to approve or reject labels. The approval is shown in the exported datasets so that downstream applications can exclude non-approved labels.

Animation showing label approving workflow

In the future, this feature can be expanded to for example include multiple reviewers per document or to provide a customized "label reviewer" role whose document sort order by default shows labeled but not-reviewed documents first in the annotation view.

@c-w c-w force-pushed the enhancement/approve-labels branch from 290894b to 8f876d4 Compare Jun 25, 2019
@c-w c-w force-pushed the enhancement/approve-labels branch from 8f876d4 to 974c2ce Compare Jun 26, 2019
@c-w c-w force-pushed the enhancement/approve-labels branch from 974c2ce to d0c566c Compare Jun 26, 2019
@Hironsan Hironsan merged commit 9d7cbfe into doccano:master Jun 28, 2019
2 checks passed
@c-w c-w deleted the enhancement/approve-labels branch Jun 28, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants