Skip to content
AlpacaTag: An Active Learning-based Crowd Annotation Framework for Sequence Tagging
Python HTML CSS JavaScript Jupyter Notebook
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.
alpaca_client Update Jul 26, 2019
alpaca_server figures Jul 27, 2019
annotation debug Jul 26, 2019
figures figures Jul 30, 2019
.gitignore setup Jul 26, 2019 Update Jul 30, 2019
_config.yml Set theme jekyll-theme-dinky Jul 24, 2019
requirements.txt bug fix Jul 17, 2019
test_server.ipynb integrating finished Jun 18, 2019


AlpacaTag is an active learning-based crowd annotation framework for sequence tagging, such as named-entity recognition (NER).

Website      Documenations      Paper      Poster

The UI framework of AlpacaTag is based on the awesome work of Doccano, while the distinctive advantages of AlpacaTag are three-fold:

  • Active intelligent recommendation: dynamically suggesting annotations and sampling the most informative unlabeled instances with a back-end active learned model.

  • Automatic crowd consolidation: enhancing real-time inter-annotator agreement by merging inconsistent labels from multiple annotators.

  • Real-time model deployment: users can deploy their models in downstream systems while new annotations are being made.

  • Overall Workflow



Annotation Tutorial

Framework Customization

Model Server APIs


     author = {Bill Yuchen Lin and Dongho Lee and Frank F. Xu and Ouyu Lan and Xiang Ren}, 
     title = {AlpacaTag: An Active Learning-based Crowd Annotation Framework for Sequence Tagging.}, 
     booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), Demo Track},
     year = {2018} 


Back-end Model APIs

Crowd Consolidation

Performance Study

You can’t perform that action at this time.