Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancement/Approve labels #258

Merged
merged 2 commits into from Jun 28, 2019

Conversation

Projects
None yet
2 participants
@c-w
Copy link
Contributor

commented Jun 25, 2019

As discussed in #249, it is a very useful feature to enable a type of users to review and approve labels that other users annotated. The review feature for example enables workflows like the ones discussed in #65 where many users can annotate the dataset and then some trusted users can verify the labels before we use them for downstream workflows such as model building.

As such, this pull request implements a simple label review mechanism: admin users now see a button on the annotation page that enables them to approve or reject labels. The approval is shown in the exported datasets so that downstream applications can exclude non-approved labels.

Animation showing label approving workflow

In the future, this feature can be expanded to for example include multiple reviewers per document or to provide a customized "label reviewer" role whose document sort order by default shows labeled but not-reviewed documents first in the annotation view.

@c-w c-w force-pushed the CatalystCode:enhancement/approve-labels branch from 290894b to 8f876d4 Jun 25, 2019

@c-w c-w force-pushed the CatalystCode:enhancement/approve-labels branch from 8f876d4 to 974c2ce Jun 26, 2019

@c-w c-w force-pushed the CatalystCode:enhancement/approve-labels branch from 974c2ce to d0c566c Jun 26, 2019

@Hironsan Hironsan merged commit 9d7cbfe into chakki-works:master Jun 28, 2019

2 checks passed

Codacy/PR Quality Review Up to standards. A positive pull request.
Details
Travis CI - Pull Request Build Passed
Details

@c-w c-w deleted the CatalystCode:enhancement/approve-labels branch Jun 28, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.