-
|
I am trying to use stanza model instead since it supported both Thai and English. I use helm chart with minikube (docker). I don't know if it's the right way but I built another image and baked in the nlp-config file like this Here's the nlp config's content Then for the analyzer config & registry config I use config map and apply with And I set the env inside Then after |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 11 replies
-
|
Hi @Khuntawee, it's difficult to know why this is crashing, but my initial hunch is that two stanza models into memory are too big for your pod. Could you share more details on your deployment? |
Beta Was this translation helpful? Give feedback.
-
|
It's hard to say what's the root cause on this. When the docker is building, can you see stanza downloading the models? |
Beta Was this translation helpful? Give feedback.
-
|
No need for the spacy-stanza package. It's outdated |
Beta Was this translation helpful? Give feedback.
-
There's stanza in the docker image. I tried the spacy-stanza package and it still faces the same problem. Here's At first i thought it was the problem because there's no spacy-stanza package since it was mentioned in the |
Beta Was this translation helpful? Give feedback.
-
|
I've tried to reproduce this, and ended up creating the config files as described in the repo. I also used .Dockerfile.stanza: FROM python:3.12-slim@sha256:f3fa41d74a768c2fce8016b98c191ae8c1bacd8f1152870a3f9f87d350920b7c
ARG NLP_CONF_FILE=presidio_analyzer/conf/default.yaml
ARG ANALYZER_CONF_FILE=presidio_analyzer/conf/default_analyzer.yaml
ARG RECOGNIZER_REGISTRY_CONF_FILE=presidio_analyzer/conf/default_recognizers.yaml
ENV PIP_NO_CACHE_DIR=1
ENV POETRY_VIRTUALENVS_CREATE=false
ENV ANALYZER_CONF_FILE=${ANALYZER_CONF_FILE}
ENV RECOGNIZER_REGISTRY_CONF_FILE=${RECOGNIZER_REGISTRY_CONF_FILE}
ENV NLP_CONF_FILE=${NLP_CONF_FILE}
ENV PORT=3000
ENV WORKERS=1
COPY ${ANALYZER_CONF_FILE} /app/${ANALYZER_CONF_FILE}
COPY ${RECOGNIZER_REGISTRY_CONF_FILE} /app/${RECOGNIZER_REGISTRY_CONF_FILE}
COPY ${NLP_CONF_FILE} /app/${NLP_CONF_FILE}
WORKDIR /app
# Install essential build tools and curl for health checks
RUN apt-get update \
&& apt-get install -y build-essential curl
COPY ./pyproject.toml /app/
RUN pip install poetry==2.3.2 && poetry install --no-root --only=main -E server -E stanza
# install nlp models specified in NLP_CONF_FILE
COPY ./install_nlp_models.py /app/
ENV STANZA_RESOURCES_DIR=/app/stanza_resources
RUN poetry run python install_nlp_models.py --conf_file ${NLP_CONF_FILE}
COPY . /app/
# Create a non-root user and set ownership
RUN useradd -m -u 1001 presidio && chown -R presidio:presidio /app
USER 1001
EXPOSE ${PORT}
HEALTHCHECK --interval=30s --timeout=3s --start-period=30s --retries=3 \
CMD curl -f http://localhost:${PORT}/health || exit 1
CMD ["./entrypoint.sh"]
Here are the config files:
supported_languages:
- en
- th
default_score_threshold: 0
supported_languages:
- en
- th
recognizers:
- name: CreditCardRecognizer
type: predefined
supported_languages:
- language: en
context: [credit, card, visa, mastercard, cc, amex, discover, jcb, diners, maestro, instapayment]
- language: th
context: [บัตรเครดิต, บัตร, วีซ่า, มาสเตอร์การ์ด]
- name: CryptoRecognizer
type: predefined
- name: DateRecognizer
type: predefined
- name: EmailRecognizer
type: predefined
- name: IbanRecognizer
type: predefined
- name: IpRecognizer
type: predefined
- name: MedicalLicenseRecognizer
type: predefined
- name: PhoneRecognizer
type: predefined
- name: UrlRecognizer
type: predefined
- name: UsBankRecognizer
type: predefined
- name: UsLicenseRecognizer
type: predefined
- name: UsItinRecognizer
type: predefined
- name: UsPassportRecognizer
type: predefined
- name: UsSsnRecognizer
type: predefined
- name: NhsRecognizer
type: predefined
- name: StanzaRecognizer
type: predefined
supported_languages:
- en
- th
- name: ThTninRecognizer
type: predefined
nlp_engine_name: stanza
models:
- lang_code: en
model_name: en
- lang_code: th
model_name: th
ner_model_configuration:
model_to_presidio_entity_mapping:
PER: PERSON
PERSON: PERSON
NORP: NRP
FAC: LOCATION
LOC: LOCATION
LOCATION: LOCATION
GPE: LOCATION
ORG: ORGANIZATION
ORGANIZATION: ORGANIZATION
DATE: DATE_TIME
TIME: DATE_TIMETo run this: # Build
docker build -f Dockerfile.stanza \
--build-arg ANALYZER_CONF_FILE=analyzer.yaml \
--build-arg NLP_CONF_FILE=nlp_engine.yaml \
--build-arg RECOGNIZER_REGISTRY_CONF_FILE=registry.yaml \
-t presidio-analyzer-stanza .
# Run
docker run -d -p 5002:3000 --name presidio-analyzer presidio-analyzer-stanza
# Test (English)
curl -X POST http://localhost:5002/analyze \
-H "Content-Type: application/json" \
-d '{"text": "My name is John Doe and my email is john@example.com", "language": "en"}'
# Test (Thai)
curl -X POST http://localhost:5002/analyze \
-H "Content-Type: application/json" \
-d '{"text": "ชื่อของฉันคือ สมชาย ใจดี และอีเมลของฉันคือ somchai@example.com", "language": "th"}' |
Beta Was this translation helpful? Give feedback.
-
|
cc @SharonHart. There's a small bug in the Dockerfile.stanza that I'll fix. There's a big issue with passing only the analyzer.conf as the default nlp_engine.conf and registry.conf are overriding it. Let's take offline. |
Beta Was this translation helpful? Give feedback.
-
|
see #1910 1910 |
Beta Was this translation helpful? Give feedback.
-
|
Thank you so much. I'll try this tmr as it is quite late over here. |
Beta Was this translation helpful? Give feedback.
I've tried to reproduce this, and ended up creating the config files as described in the repo. I also used
dockerfile.stanza(with one small change) instead of the custom dockerfile that you provided. I was able to build it locally (it also downloaded the English and Thai stanza models during build)..Dockerfile.stanza: