CloudSearch Domain Resource Support

[celestialorb]

Community Note

• Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Closes #7833

Output from acceptance testing:

make testacc TESTARGS="-run TestAccAWSCloudSearchDomain"
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run TestAccAWSCloudSearchDomain -timeout 120m
=== RUN   TestAccAWSCloudSearchDomain_basic
=== PAUSE TestAccAWSCloudSearchDomain_basic
=== RUN   TestAccAWSCloudSearchDomain_textAnalysisScheme
=== PAUSE TestAccAWSCloudSearchDomain_textAnalysisScheme
=== RUN   TestAccAWSCloudSearchDomain_badName
=== PAUSE TestAccAWSCloudSearchDomain_badName
=== RUN   TestAccAWSCloudSearchDomain_badInstanceType
=== PAUSE TestAccAWSCloudSearchDomain_badInstanceType
=== RUN   TestAccAWSCloudSearchDomain_badIndexFieldNames
=== PAUSE TestAccAWSCloudSearchDomain_badIndexFieldNames
=== RUN   TestAccAWSCloudSearchDomain_badIndexFieldType
=== PAUSE TestAccAWSCloudSearchDomain_badIndexFieldType
=== CONT  TestAccAWSCloudSearchDomain_basic
=== CONT  TestAccAWSCloudSearchDomain_badIndexFieldNames
=== CONT  TestAccAWSCloudSearchDomain_badIndexFieldType
=== CONT  TestAccAWSCloudSearchDomain_badName
=== CONT  TestAccAWSCloudSearchDomain_badInstanceType
=== CONT  TestAccAWSCloudSearchDomain_textAnalysisScheme
--- PASS: TestAccAWSCloudSearchDomain_badInstanceType (2.85s)
--- PASS: TestAccAWSCloudSearchDomain_badIndexFieldType (3.01s)
--- PASS: TestAccAWSCloudSearchDomain_badName (3.04s)
--- PASS: TestAccAWSCloudSearchDomain_badIndexFieldNames (4.18s)
--- PASS: TestAccAWSCloudSearchDomain_textAnalysisScheme (41.59s)
--- PASS: TestAccAWSCloudSearchDomain_basic (60.54s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       60.616s

[github-actions]

Thank you for your contribution! 🚀

Please note that typically Go dependency changes are handled in this repository by dependabot or the maintainers. This is to prevent pull request merge conflicts and further delay reviews of contributions. Remove any changes to the go.mod or go.sum files and commit them into this pull request.

Additional details:

• Check open pull requests with the dependencies label to view other dependency updates.
• If this pull request includes an update the AWS Go SDK (or any other dependency) version, only updates submitted via dependabot will be merged. This pull request will need to remove these changes and will need to be rebased after the existing dependency update via dependabot has been merged for this pull request to be reviewed.
• If this pull request is for supporting a new AWS service:
  • Ensure the new AWS service changes are following the Contributing Guide section on new services, in particular that the dependency addition and initial provider support are in a separate pull request from other changes (e.g. new resources). Contributions not following this item will not be reviewed until the changes are split.
  • If this pull request is already a separate pull request from the above item, you can ignore this message.

[github-actions]

Welcome @celestialorb 👋

It looks like this is your first Pull Request submission to the Terraform AWS Provider! If you haven’t already done so please make sure you have checked out our CONTRIBUTING guide and FAQ to make sure your contribution is adhering to best practice and has all the necessary elements in place for a successful approval.

Also take a look at our FAQ which details how we prioritize Pull Requests for inclusion.

Thanks again, and welcome to the community! 😃

[celestialorb]

Only question I have at the moment from someone more intimately familiar with Terraform resource/provider design, is should an index in a CloudSearch domain be a separate resource? I feel as if it should be due to its complexity, but perhaps the initial approach taken by @Mi3-14159 is perfectly fine.

[austinkelleher]

Nice work @celestialorb! This would be extremely helpful to our team! Is there something blocking the review of this PR?

[celestialorb]

@austinkelleher Not sure, haven't seen anyone official pop into the PR yet. Don't know what can be done to get it noticed; I'm sure there's a ton of PRs to review on their end. I've gone ahead and updated the code and docs so that all the checks pass again for this PR.

In the meantime I've published this custom version of the provider under my own namespace, celestialorb/aws. If you use that as your source in your Terraform it should pull this version you'll be able to use it for CloudSearch domains.

[hallsamuel90]

Awesome job @celestialorb !! It's working great and couldn't have come at a better time for our team!

[vsharma29]

@celestialorb Thanks for the provider. I am getting an issue even if i dont change the index elements still the plan shows me changes if i do terraform plan and it recreates all of the index everytime.

resource "aws_cloudsearch_domain" "domain" {
name = var.domain_name
instance_type = var.instance_type
multi_az = true

dynamic "index" {
for_each = var.indexes
content {
name = index.value["name"]
type = index.value["type"]
search = lookup(index, "search", true)
facet = lookup(index, "facet", true)
return = lookup(index, "return", true)
sort = lookup(index, "sort", true)
highlight = lookup(index, "highlight", true)
analysis_scheme = lookup(index, "analysis_scheme", "en_default")
}
}
tags = var.tags

service_access_policies = data.aws_iam_policy_document.cloudsearch_access_policy.json
}

Value from vars file

indexes = [
{
name = "abstract"
type = "text"
},
{
name = "analystname"
type = "text-array"
},
{
name = "htmtext"
type = "text"
},
{
name = "issuername"
type = "text"
},
{
name = "organizationid"
type = "text"
},
{
name = "pdftext"
type = "text"
},
{
name = "reporttype"
type = "text"
},
{
name = "sectorlevel1"
type = "text"
},
{
name = "sectorlevel2"
type = "text"
},
{
name = "subtitle"
type = "text"
},
{
name = "ticker"
type = "text"
},
{
name = "title"
type = "text"
},
{
name = "url"
type = "text"
}
]

[celestialorb]

@celestialorb Thanks for the provider. I am getting an issue even if i dont change the index elements still the plan shows me changes if i do terraform plan and it recreates all of the index everytime.

resource "aws_cloudsearch_domain" "domain" {
  name = var.domain_name
  instance_type = var.instance_type
  multi_az = true

  dynamic "index" {
    for_each = var.indexes
    content {
      name = index.value["name"]
      type = index.value["type"]
      search = lookup(index, "search", true)
      facet = lookup(index, "facet", true)
      return = lookup(index, "return", true)
      sort = lookup(index, "sort", true)
      highlight = lookup(index, "highlight", true)
      analysis_scheme = lookup(index, "analysis_scheme", "_en_default_")
    }
  }

  tags = var.tags

  service_access_policies = data.aws_iam_policy_document.cloudsearch_access_policy.json
}

Value from vars file

indexes = [
{
  name = "abstract"
  type = "text"
},
{
  name = "analystname"
  type = "text-array"
},
{
  name = "htmtext"
  type = "text"
},
{
  name = "issuername"
  type = "text"
},
{
  name = "organizationid"
  type = "text"
},
{
  name = "pdftext"
  type = "text"
},
{
  name = "reporttype"
  type = "text"
},
{
  name = "sectorlevel1"
  type = "text"
},
{
  name = "sectorlevel2"
  type = "text"
},
{
  name = "subtitle"
  type = "text"
},
{
  name = "ticker"
  type = "text"
},
{
  name = "title"
  type = "text"
},
{
  name = "url"
  type = "text"
}
]

@vsharma29 The issue here is that you're setting an analysis_scheme for indices where one doesn't make sense (and can't be set). Analysis schemes are only valid for text and text-array fields, but in your Terraform configuration you're setting the analysis_scheme for all indices via the lookup function:

analysis_scheme = lookup(index, "analysis_scheme", "_en_default_")

There's still some work to done surrounding indices, given that they aren't exactly straight-forward and I think the approach originally taken can be improved upon.

[vsharma29]

@celestialorb All the fields which i have set up are text or text array so it can be set . I am not sure whats driving the plan to
Misbehave but something wierd going on

[celestialorb]

@celestialorb All the fields which i have set up are text or text array so it can be set . I am not sure whats driving the plan to
Misbehave but something wierd going on

My mistake, I thought I had spotted an int in there somewhere. I was able to reproduce the issue earlier when using non-text fields like int, but not with only text fields. I'll go back and take another look later.

[celestialorb]

@celestialorb All the fields which i have set up are text or text array so it can be set . I am not sure whats driving the plan to
Misbehave but something wierd going on

Including a copy of the plan would help to debug the issue.

[celestialorb]

@celestialorb All the fields which i have set up are text or text array so it can be set . I am not sure whats driving the plan to
Misbehave but something wierd going on

Yeah, I'll need to see what your plan looks like. I can't reproduce the issue on my end.

[vsharma29]

Sorry about late reply. Copy of my plan.`
pipelineLog-10.txt

[celestialorb]

Sorry about late reply. Copy of my plan.`
pipelineLog-10.txt

Looks like it's because you're changing the search attribute from false to true on a lot of these indices, if not all, so it's recreating them. This is expected behavior.

[zhelding]

Pull request #21306 has significantly refactored the AWS Provider codebase. As a result, most PRs opened prior to the refactor now have merge conflicts that must be resolved before proceeding.

Specifically, PR #21306 relocated the code for all AWS resources and data sources from a single aws directory to a large number of separate directories in internal/service, each corresponding to a particular AWS service. This separation of code has also allowed for us to simplify the names of underlying functions -- while still avoiding namespace collisions.

We recognize that many pull requests have been open for some time without yet being addressed by our maintainers. Therefore, we want to make it clear that resolving these conflicts in no way affects the prioritization of a particular pull request. Once a pull request has been prioritized for review, the necessary changes will be made by a maintainer -- either directly or in collaboration with the pull request author.

For a more complete description of this refactor, including examples of how old filepaths and function names correspond to their new counterparts: please refer to issue #20000.

For a quick guide on how to amend your pull request to resolve the merge conflicts resulting from this refactor and bring it in line with our new code patterns: please refer to our Service Package Refactor Pull Request Guide.

[ewbankkit]

LGTM 🚀.

Commercial

% make testacc TESTARGS='-run=TestAccCloudSearchDomain_' PKG=cloudsearch 
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/cloudsearch/... -v -count 1 -parallel 20 -run=TestAccCloudSearchDomain_ -timeout 180m
=== RUN   TestAccCloudSearchDomain_basic
=== PAUSE TestAccCloudSearchDomain_basic
=== RUN   TestAccCloudSearchDomain_disappears
=== PAUSE TestAccCloudSearchDomain_disappears
=== RUN   TestAccCloudSearchDomain_indexFields
=== PAUSE TestAccCloudSearchDomain_indexFields
=== RUN   TestAccCloudSearchDomain_update
=== PAUSE TestAccCloudSearchDomain_update
=== CONT  TestAccCloudSearchDomain_basic
=== CONT  TestAccCloudSearchDomain_indexFields
=== CONT  TestAccCloudSearchDomain_disappears
=== CONT  TestAccCloudSearchDomain_update
--- PASS: TestAccCloudSearchDomain_disappears (662.90s)
--- PASS: TestAccCloudSearchDomain_basic (679.61s)
--- PASS: TestAccCloudSearchDomain_indexFields (1588.94s)
--- PASS: TestAccCloudSearchDomain_update (1789.34s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/cloudsearch        1792.704s
% make testacc TESTARGS='-run=TestAccCloudSearchDomainServiceAccessPolicy_' PKG=cloudsearch
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/cloudsearch/... -v -count 1 -parallel 20 -run=TestAccCloudSearchDomainServiceAccessPolicy_ -timeout 180m
=== RUN   TestAccCloudSearchDomainServiceAccessPolicy_basic
=== PAUSE TestAccCloudSearchDomainServiceAccessPolicy_basic
=== RUN   TestAccCloudSearchDomainServiceAccessPolicy_update
=== PAUSE TestAccCloudSearchDomainServiceAccessPolicy_update
=== CONT  TestAccCloudSearchDomainServiceAccessPolicy_basic
=== CONT  TestAccCloudSearchDomainServiceAccessPolicy_update
--- PASS: TestAccCloudSearchDomainServiceAccessPolicy_basic (1807.86s)
--- PASS: TestAccCloudSearchDomainServiceAccessPolicy_update (2546.93s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/cloudsearch        2550.273s

GovCloud

% make testacc TESTARGS='-run=TestAccCloudSearchDomain_' PKG=cloudsearch
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/cloudsearch/... -v -count 1 -parallel 20 -run=TestAccCloudSearchDomain_ -timeout 180m
=== RUN   TestAccCloudSearchDomain_basic
=== PAUSE TestAccCloudSearchDomain_basic
=== RUN   TestAccCloudSearchDomain_disappears
=== PAUSE TestAccCloudSearchDomain_disappears
=== RUN   TestAccCloudSearchDomain_indexFields
=== PAUSE TestAccCloudSearchDomain_indexFields
=== RUN   TestAccCloudSearchDomain_update
=== PAUSE TestAccCloudSearchDomain_update
=== CONT  TestAccCloudSearchDomain_basic
=== CONT  TestAccCloudSearchDomain_indexFields
=== CONT  TestAccCloudSearchDomain_disappears
=== CONT  TestAccCloudSearchDomain_update
=== CONT  TestAccCloudSearchDomain_basic
    acctest.go:608: skipping tests; partition aws-us-gov does not support cloudsearch service
=== CONT  TestAccCloudSearchDomain_indexFields
    acctest.go:608: skipping tests; partition aws-us-gov does not support cloudsearch service
--- SKIP: TestAccCloudSearchDomain_indexFields (1.47s)
--- SKIP: TestAccCloudSearchDomain_basic (1.47s)
=== CONT  TestAccCloudSearchDomain_disappears
    acctest.go:608: skipping tests; partition aws-us-gov does not support cloudsearch service
--- SKIP: TestAccCloudSearchDomain_disappears (1.47s)
=== CONT  TestAccCloudSearchDomain_update
    acctest.go:608: skipping tests; partition aws-us-gov does not support cloudsearch service
--- SKIP: TestAccCloudSearchDomain_update (1.47s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/cloudsearch        4.968s
% make testacc TESTARGS='-run=TestAccCloudSearchDomainServiceAccessPolicy_' PKG=cloudsearch
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/cloudsearch/... -v -count 1 -parallel 20 -run=TestAccCloudSearchDomainServiceAccessPolicy_ -timeout 180m
=== RUN   TestAccCloudSearchDomainServiceAccessPolicy_basic
=== PAUSE TestAccCloudSearchDomainServiceAccessPolicy_basic
=== RUN   TestAccCloudSearchDomainServiceAccessPolicy_update
=== PAUSE TestAccCloudSearchDomainServiceAccessPolicy_update
=== CONT  TestAccCloudSearchDomainServiceAccessPolicy_basic
=== CONT  TestAccCloudSearchDomainServiceAccessPolicy_update
=== CONT  TestAccCloudSearchDomainServiceAccessPolicy_basic
    acctest.go:608: skipping tests; partition aws-us-gov does not support cloudsearch service
--- SKIP: TestAccCloudSearchDomainServiceAccessPolicy_basic (1.30s)
=== CONT  TestAccCloudSearchDomainServiceAccessPolicy_update
    acctest.go:608: skipping tests; partition aws-us-gov does not support cloudsearch service
--- SKIP: TestAccCloudSearchDomainServiceAccessPolicy_update (1.30s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/cloudsearch        4.651s

[ewbankkit]

@celestialorb Thanks for the contribution 🎉 👏.
I made some effectively cosmetic changes (index -> index_field and added some additional configuration options) to aws_cloudsearch_domain and split the policy into its own resource aws_cloudsearch_domain_service_access_policy.

[github-actions]

This functionality has been released in v3.73.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.