From 17d4ba04bb078fd904b5b37f8253bc891727c5d5 Mon Sep 17 00:00:00 2001 From: Mike Birnstiehl <114418652+mdbirnstiehl@users.noreply.github.com> Date: Fri, 5 Sep 2025 11:41:47 -0500 Subject: [PATCH] Remove JSON malformation risk in anonymization limitations https://github.com/elastic/docs-content/issues/2695 --- docs/en/observability/observability-ai-assistant.asciidoc | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/en/observability/observability-ai-assistant.asciidoc b/docs/en/observability/observability-ai-assistant.asciidoc index 83f9d5d85a..ae19602fbd 100644 --- a/docs/en/observability/observability-ai-assistant.asciidoc +++ b/docs/en/observability/observability-ai-assistant.asciidoc @@ -520,7 +520,6 @@ Anonymization has the following limitations: * *Performance (NER)*: Running an NER model can add latency depending on the request. To improve performance of the model, consider scaling up your ML nodes by adjusting deployment parameters: increase `number_of_allocations` for better throughput and `threads_per_allocation` for faster individual requests. For details, refer to https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-start-trained-model-deployment[start trained model deployment API]. * *Structured JSON*: The NER model we validated (`elastic/distilbert-base-uncased-finetuned-conll03-english`) is trained on natural English text and often misses entities inside JSON or other structured data. If thorough masking is required, prefer regex rules and craft them to account for JSON syntax. * *False negatives / positives*: No model or pattern is perfect. Model accuracy may vary depending on model and input. -* *JSON malformation risk*: Both NER inference and regex rules can potentially create malformed JSON when anonymizing JSON data such as function responses. This can occur by replacing text across character boundaries, which may break JSON structure causing the whole request to fail. If this occurs, you may need to adjust your regex pattern or disable the NER rule. [discrete]