EnterpriseSearchRestIT failure #95917
Description
Context
With #95775, enterprise search adds the behavioral_analytics-events-final_pipeline lazily only when the data stream is created. This contradicts a validation that is about to be added within #95782, which does not allow to install an index template if it refers to a non-existing default_pipeline or final_pipeline.
As agreed with @jimczi, we will disable EnterpriseSearchRestIT for now, so that it doesn't block #95782 and figure out how to handle that later on.
One optional solution we discussed is changing behavioral_analytics-events-final_pipeline to contain:
{
"processors": [
{
"pipeline" : {
"name": "lazily-added",
"ignore_missing_pipeline": true,
"description": "The actual pipeline, which is installed lazily"
}
}
]
}
and install the lazily-added upon the proper trigger (data stream creation or other).
Note that while this will prevent dependency-related errors, it may still imply there is an undesirable race condition between the installation of the ingest pipeline and the referring index template.
CI Link
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+pull-request+part-2/33583/console
Repro line
./gradlew ':x-pack:plugin:ent-search:qa:rest:yamlRestTest' --tests "org.elasticsearch.xpack.entsearch.EnterpriseSearchRestIT.test {p0=entsearch/80_behavioral_analytics_delete/Delete Analytics Collection - Analytics Collection does not exist}" -Dtests.seed=714C8C6B612820A4
Does it reproduce?
Yes
Applicable branches
eyalkoren:ingest-pipeline-registry
Failure history
No response
Failure excerpt
"type" : "status_exception",
"reason" : "error while creating analytics collection [my-test-analytics-collection]",
"caused_by" : {
"type" : "illegal_argument_exception",
"reason" : "no matching index template found for data stream [behavioral_analytics-events-my-test-analytics-collection]",
"stack_trace" : "java.lang.IllegalArgumentException: no matching index template found for data stream [behavioral_analytics-events-my-test-analytics-collection] |
| at org.elasticsearch.server@8.9.0-SNAPSHOT/org.elasticsearch.cluster.metadata.MetadataCreateDataStreamService.lookupTemplateForDataStream(MetadataCreateDataStreamService.java:307) |
| at org.elasticsearch.server@8.9.0-SNAPSHOT/org.elasticsearch.cluster.metadata.MetadataCreateDataStreamService.createDataStream(MetadataCreateDataStreamService.java:221) |
| at org.elasticsearch.server@8.9.0-SNAPSHOT/org.elasticsearch.cluster.metadata.MetadataCreateDataStreamService.createDataStream(MetadataCreateDataStreamService.java:168) |
...