[BUG] Security Analytics Detector creation overwrites Data-Stream template Index-Patterns in OpenSearch

[rafaelma]

What is the bug?

When creating a Security Analytics Detector with a data-stream "Data source" in OpenSearch, the index-patterns defined in the data-stream template is being overwritten. The creation of the detector modifies the existing index pattern to match the specific data stream used by the detector and adds a new link to a .opensearch-sap-alias-mappings-component... component_template. This action renders the template unusable for other data streams that were designed to match the original index pattern.

How can one reproduce the bug?

Steps to reproduce the behavior:

1. Define a data-stream template with an index pattern, e.g., "ss4o_logs-dns-bind9.querylog*".
2. Define a data stream named "ss4o_logs-dns_bind9.querylog-prod" to use the aforementioned template.
3. Define another data stream named "ss4o_logs-dns_bind9.querylog-test" that uses the same template.
4. Create a Security Analytics Detector for the production environment, with "Data Source" set to "ss4o_logs-dns_bind9.querylog-prod".
5. Inspect the data-stream template after the detector creation.
6. Notice that the original index-pattern "ss4o_logs-dns-bind9.querylog*" has been changed to "ss4o_logs-dns-bind9.querylog-prod*".
7. Additionally, identify that the template has acquired a new link to a component template named ".opensearch-sap-alias-mappings-component-ss4o_logs-dns_bind9.querylog-prod".
8. Realize that the data-stream template is no longer valid for "ss4o_logs-dns_bind9.querylog-test" due to the index pattern modification.

What is the expected behavior?

Creating a Security Analytics Detector should link a new component template to the template used by the "Data Source" without modifying the existing index pattern in the template. This would allow the template to remain valid for all data streams matching the original index pattern design.

Actual Result: The creation of the Security Analytics Detector causes an alteration in the data-stream template index pattern, from "ss4o_logs-dns-bind9.querylog*" in our example to a more specific pattern matching the production data stream. In effect, this breaks the template for any other data stream that was supposed to use the same pattern, such as "ss4o_logs-dns_bind9.querylog-test".

What is your host/environment?

• OS: Red Hat Enterprise Linux release 9.4
• Version Opensearch 2.14
• Plugins

 # /usr/share/opensearch/bin/opensearch-plugin list
opensearch-alerting
opensearch-anomaly-detection
opensearch-asynchronous-search
opensearch-cross-cluster-replication
opensearch-custom-codecs
opensearch-flow-framework
opensearch-geospatial
opensearch-index-management
opensearch-job-scheduler
opensearch-knn
opensearch-ml
opensearch-neural-search
opensearch-notifications
opensearch-notifications-core
opensearch-observability
opensearch-performance-analyzer
opensearch-reports-scheduler
opensearch-security
opensearch-security-analytics
opensearch-skills
opensearch-sql

Do you have any screenshots?
If applicable, add screenshots to help explain your problem.

Do you have any additional context?

This issue occurs only if the .opensearch-sap-alias-mappings-component* component template doesn't exist, and its reference is not already included in the data-stream template. The system should ensure that any new configurations added to cater to the detector do not negatively impact the underlying template structures and their reusability.

The code responsible for this behavior is probably:

• https://github.com/opensearch-project/security-analytics/blob/main/src/main/java/org/opensearch/securityanalytics/mapper/IndexTemplateManager.java#L184
• https://github.com/opensearch-project/security-analytics/blob/main/src/main/java/org/opensearch/securityanalytics/mapper/IndexTemplateManager.java#L219

[dblock]

[Catch All Triage - 1, 2, 3]