The fastent Python library is a tool for end-to-end creation of custom models for named-entity recognition.
To train a model for a new type of entity, you just need a list of examples.
You are not limited to only predefined types like person, location and organization.
How It Works
fastent does end-to-end creation: dataset generation, annotation, contextualiziation and training a model.
You can also use fastent modules as standalone tools.
Made for Prod
fastent includes integrations with tools like spaCy, fastText pre-trained models and NLTK.
fastent is built to scale to very large text datasets in many languages.
- How To
fastent is developed for Python 3 on Unix systems.
Clone this repo or install from PyPI:
pip install fastent
Download NLTK data:
python -m nltk.downloader stopwords
Install and set up CouchDB:
wget -O - https://raw.githubusercontent.com/fastent/fastent/master/install.sh | bash
Downloading data files
TODO: fastText stuff
fastent can generate a dataset from a list
fastent can even generate a list from one or two examples.
from fastent import dataset_pseudo_generator model = dataset_pseudo_generator.spacy_initialize('en_core_web_lg') dataset_pseudo_generator.dataset_generate(model,['cocaine', 'heroin'], 100)
The equivalent on the command line:
python dataset_pseudo_generator.py -m en_core_web_lg -s cocaine,heroin
To train a model from the annotated and contextualized dataset:
For now the only supported learning framework is spaCy.
TODO: sample output
fastent includes integrations for downloading datasets and pre-trained models.
See how fastent performs on benchmarks
Browse frequently asked questions