AWS Opensearch ISM support

[ellisroll-b]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

With AWS supporting index state management in OpenSearch, many of us could remove home grown processing to manage older data. The new ISM functionality supports additional AWS hosted functionality supports a number of features that are attractive, many would be desirable to configure at deployment.

New or Affected Resource(s)

aws_elasticsearch_domain

Potentially new resources for ISM configuration, as your design sees fit.

References

https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ism.html

• #0000

[jackson-theisen]

While I would be grateful for Terraform-managed ISM policies, I think the bigger issue at hand is interacting with OpenSearch APIs in general. There are a number of other API-managed components of a domain that don't have Terraform support, which results in a lot of 'home grown processing' (as @ellisroll-b said) for folks who prefer an everything-as-code approach. Examples would be index templates, audit/compliance configuration, monitor templates, roles/role mappings, etc. While https://github.com/phillbaker/terraform-provider-elasticsearch does a great job for many of these tasks, it's not built specifically for Amazon OpenSearch Service. IMO, the AWS provider needs to add support for HTTP requests to Amazon OpenSearch Service. With so many nuances imposed by the AWS managed service, we need guard rails and validation in place to guide developers into best practices for interacting with their domain. Here's a few examples of aforementioned nuances that I feel makes it worthwhile to add the functionality discussed (i'm sure there are a bunch more):

• If your domain access policy includes IAM users or roles (or you use an IAM master user with fine-grained access control), you must sign requests to the OpenSearch APIs with your IAM credentials.
• Various types of logs (application, audit, index slow, etc.) have CloudWatch dependencies.
• Only a subset of ISM operations are supported.
• Enabling plugins

[jackson-theisen]

I have also opened phillbaker/terraform-provider-elasticsearch#299 and opensearch-project/opensearch-devops#82 which I feel would be worth coordinating with some of the maintainers on to determine what pieces belong where.