Support for declarative/IaC Elasticsearch API requests

[brsolomon-deloitte]

Proposal

Allow some initial request to the Elasticsearch API to be declared with IaC through either CRDs or Helm chart parameters.

Use case. Why is this important?

From an administrative standpoint, deployment of Elastic stack does not stop once the Elasticsearch cluster itself has been provisioned. Initial steps also commonly consist of creating things like component/index template, ingest pipelines, ILM policies.

Currently we do these through the Dev Tools console in Kibana. For example,

PUT _component_template/logs-xxx-yyy
{ ... }
PUT _ingest/pipeline/logs-xxx-yyy-pipeline
{ ... }
PUT _component_template/logs-xxx-yyy-settings
{ ... }
PUT /_index_template/logs-xxx-yyy
{ ... }

This violates our GitOps git-as-source-of-truth paradigm since it creates a new cluster state that is not captured in our git-sourced Helm charts.

Since we anticipate multiple clusters taking these same initial index management components, it would be great to have a formal way to declare them alongside the Elastic stack parameters themselves.

Boiling things down, pretty much all API requests consist of:

• method
• URL path
• optional body

It seems like that is something that could potentially be parameterized alongside the initial deployment. Spitballing here:

initialApiObjects:
  - method: PUT
    path: _component_template/logs-xxx-yyy
    payload: |
      {"foo": "bar"}

[barkbay]

Hi, I think this is a duplicate of #3598. Feel free to reopen or comment if needed.

Thanks