Bothub NLP - Natural Language Processing services

Check the main Bothub project repository.

Services

bothub-nlp-nlu-worker-on-demand

Packages

bothub-nlp (python 3.6)

bothub-nlp-celery (python 3.6)

bothub-nlp-nlu (python 3.6)

Requirements

• Python (3.6)
• Docker
• Docker-Compose

Development

Use make commands to lint, init_env, start_development.

Command	Description
make lint	Show lint warnings and errors
make init_env	Init file .env with variables environment
make start_development	Create .env with variable environment and start build docker

Environment Variables

You can set environment variables in your OS, write on .env file or pass via Docker config.

Variable	Type	Default	Description
BOTHUB_NLP_DOCKER_CLIENT_BASE_URL	string	unix://var/run/docker.sock	Docker Client Rest API URL.
BOTHUB_NLP_NLU_WORKER_ON_DEMAND_API_PORT	int	2658	Port to server API webservice.
BOTHUB_NLP_NLU_WORKER_DOCKER_IMAGE_NAME	string	ilha/bothub-nlp-nlu-worker	Bothub NLP NLU Worker Docker image.
BOTHUB_NLP_NLU_WORKER_ON_DEMAND_DOWN_TIME	int	10	Down worker after x minutes without interaction.
BOTHUB_NLP_NLU_WORKER_ON_DEMAND_NETWORKS	List separated by common.	bothub-nlp	Networks to assign in new worker service.
BOTHUB_NLP_NLU_WORKER_ON_DEMAND_RUN_IN_WORKER_NODE	boolean	False	Run service just in Workers Nodes in Docker Swarm cluster.
BOTHUB_NLP_NLU_WORKER_ON_DEMAND_CONFIG_FILE	string	bothub-nlp-nlu-worker-on-demand.cfg
BOTHUB_NLP_NLU_WORKER_ON_DEMAND_API_BASIC_AUTHORIZATION	string	None	Fill api_basic_authorization kwarg in CeleryWorkerOnDemand constructor
SUPPORTED_LANGUAGES	str	`en	pt`
BOTHUB_ENGINE_URL	str	https://api.bothub.it	Web service url
BOTHUB_NLP_CELERY_BROKER_URL	str	redis://localhost:6379/0	Celery Broker URL, check usage instructions in Celery Docs
BOTHUB_NLP_CELERY_BACKEND_URL	str	BOTHUB_NLP_CELERY_BROKER_URL value	Celery Backend URL, check usage instructions in Celery Docs
BOTHUB_NLP_NLU_AGROUP_LANGUAGE_QUEUE	boolean	True	Agroup tasks by language in celery queue, if True there will be only one queue per language.
BOTHUB_NLP_AWS_S3_BUCKET_NAME	str
BOTHUB_NLP_AWS_ACCESS_KEY_ID	str
BOTHUB_NLP_AWS_SECRET_ACCESS_KEY	str
BOTHUB_NLP_AWS_REGION_NAME	str
AWS_CLI	boolean	True	Enable install AWS CLI
AWS_ACCESS_KEY_ID	str	None	Key AWS ACCESS KEY ID
AWS_SECRET_ACCESS_KEY	str	None	Key AWS SECRET KEY
AWS_DEFAULT_REGION	str	None	DEFAULT REGION AWS
BOTHUB_SERVICE	str	docker	Choose kubernetes or docker to boot on demand
ENVIRONMENT	str	production	Choose production or staging
BOTHUB_NLU_VERSION	str	None	Choose the version of the nlp worker, if you do not define a specific version, on-demand will work without versioning the docker images
BOTHUB_NLU_CELERY_SCALE	str	5,3	Set the total workers of the celery auto scale

Docker Arguments

You need to set --build-arg when you are building docker-compose

Argument	Type	Default	Description
DOWNLOAD_SPACY_MODELS	string	en:en_core_web_md	Set supported languages. Separe languages using |. You can set location follow the format: [LANGUAGE_CODE]:[LANGUAGE_LOCATION].