Injecting timestamp in index name that is not a suffix throws IllegalArgumentException

[graytaylor0]

Describe the issue
When using the OpenSearch sink with timestamp formats such as index-%{yyyyMMdd}-test, the OpenSearch sink throws an IllegalArgumentException on startup after validating that this format is being injected in a suffix such as index-test-%{yyyyMMdd}

2024-01-11T18:56:53,018 [log-pipeline-sink-worker-2-thread-1] ERROR org.opensearch.dataprepper.pipeline.common.PipelineThreadPoolExecutor - Pipeline [log-pipeline] process worker encountered a fatal exception, cannot proceed further
java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: Time pattern can only be a suffix of an index.
	at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:?]
	at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:?]

There is no reason/limitation that requires this timestamp injection to be as a suffix, so injection of this timestamp into the index at any point should be supported.

This validation occurs here (

data-prepper/data-prepper-plugins/opensearch/src/main/java/org/opensearch/dataprepper/plugins/sink/opensearch/index/AbstractIndexManager.java

Line 138 in 787064e

if (!indexAlias.endsWith(timePattern + "}")) {

), and should be removed. Additionally, the formatter should be expanded to not be specific to suffixes like it is here (

data-prepper/data-prepper-plugins/opensearch/src/main/java/org/opensearch/dataprepper/plugins/sink/opensearch/index/AbstractIndexManager.java

Line 175 in 787064e

final String formattedTimeString = indexTimeSuffixFormatter.get()

)

Additional context
Add any other context about the problem here.

[dlvenable]

OpenSearch does support index patterns with wildcards in the middle. So this should be possible.

However, we need to check various parts of our code. We currently have some code scattered throughout the opensearch sink which assumes the wildcard is at the end. In particular, this is how we determine the alias. We might want to think about what the right alias would be for such a pipeline.

With the example above (index-%{yyyyMMdd}-test), would the index alias be index or index-test? I tend to think the latter.