Terraform crashes on trying to apply s3 bucket inventory configs

[ghost]

This issue was originally opened by @emagutu as hashicorp/terraform#27396. It was migrated here as a result of the provider split. The original body of the issue is below.

---

Terraform Version

Terraform v0.14.3
+ provider registry.terraform.io/hashicorp/aws v3.22.0

Terraform Configuration Files

  aws_region = "us-east-1"
  buckets = {
    baas-s3-broker-aws-us-east-1-prod-iac = {
      acl                   = null
      kms_master_key_alias  = null
      logging_target_bucket = "baas-s3-broker-aws-us-east-1-prod-logs-iac"
      logging_target_prefix = "logs/"
      name                  = "baas-s3-broker-aws-us-east-1-prod-iac"
      sse_algorithm         = null
      versioning            = true
    }
    baas-s3-broker-aws-us-east-1-prod-logs-iac = {
      acl                   = "log-delivery-write"
      kms_master_key_alias  = null
      logging_target_bucket = null
      logging_target_prefix = null
      name                  = "baas-s3-broker-aws-us-east-1-prod-logs-iac"
      sse_algorithm         = null
      versioning            = null
    }
    binstore-statistics-use1-iac = {
      acl                   = "private"
      kms_master_key_alias  = null
      logging_target_bucket = null
      logging_target_prefix = null
      name                  = "binstore-statistics-use1-iac"
      sse_algorithm         = null
      versioning            = null
    }
  }
  bucket_policies = {
    baas-s3-broker-aws-us-east-1-prod-iac = {
      name = null
      policy_statements = [{
        actions = ["s3:PutObject"]
        conditions = [{
          test     = "StringNotEquals"
          values   = ["aws:kms"]
          variable = "s3:x-amz-server-side-encryption"
        }]
        effect = "Deny"
        principal = {
          aws     = ["*"]
          service = []
        }
        resources = ["arn:aws:s3:::baas-s3-broker-aws-us-east-1-prod-iac/*"]
        }, {
        actions = ["s3:PutObject"]
        conditions = [{
          test     = "Null"
          values   = ["true"]
          variable = "s3:x-amz-server-side-encryption"
        }]
        effect = "Deny"
        principal = {
          aws     = ["*"]
          service = []
        }
        resources = ["arn:aws:s3:::baas-s3-broker-aws-us-east-1-prod-iac/*"]
        }, {
        actions = ["s3:PutObject"]
        conditions = [{
          test     = "Null"
          values   = ["true"]
          variable = "s3:x-amz-server-side-encryption"
        }]
        effect = "Deny"
        principal = {
          aws     = ["*"]
          service = []
        }
        resources = ["arn:aws:s3:::baas-s3-broker-aws-us-east-1-prod-iac/*"]
      }]
    }
  }
  buckets_analytics = {
    baas-s3-broker-aws-us-east-1-prod-iac = {
      analytics = {
        cloudwiry-analytics = {
          bucket_account_id     = "282711413064"
          bucket_arn            = "arn:aws:s3:::s3-reports-cloudwiry-us-east-1"
          destination_prefix    = "DevFactory"
          filter_prefix         = null
          format                = null
          name                  = "cloudwiry-analytics"
          output_schema_version = null
          tags                  = {}
        }
        rehau = {
          bucket_account_id     = null
          bucket_arn            = null
          destination_prefix    = null
          filter_prefix         = "56ab5129-f171-431a-87ca-4a80568cbeb6"
          format                = null
          name                  = "rehau"
          output_schema_version = null
          tags                  = {}
        }
      }
      name = "baas-s3-broker-aws-us-east-1-prod-iac"
    }
  }
  bucket_notifications = {
    baas-s3-broker-aws-us-east-1-prod-iac = {
      lambda_notification = [{
        events              = ["s3:ObjectCreated:*"]
        filter_prefix       = null
        filter_suffix       = null
        id                  = null
        lambda_function_arn = "arn:aws:lambda:us-east-1:111111111111:function:increment-binstore-size-aws-us-east-1-ms-prod-iac"
        }, {
        events              = ["s3:ObjectRemoved:*"]
        filter_prefix       = null
        filter_suffix       = null
        id                  = null
        lambda_function_arn = "arn:aws:lambda:us-east-1:111111111111:function:increment-binstore-size-aws-us-east-1-ms-prod-iac"
      }]
      name = null
    }
  }
  buckets_inventory = {
    baas-s3-broker-aws-us-east-1-prod-iac = {
      account_id               = null
      bucket_arn               = "arn:aws:s3:::binstore-statistics-use1-iac"
      bucket_encryption        = null
      bucket_format            = "CSV"
      bucket_prefix            = null
      destination_bucket       = null
      enabled                  = null
      encryption_sse_kms       = null
      encryption_sse_s3        = null
      filter_prefix            = null
      included_object_versions = "All"
      name                     = "binstore-statistics-use1"
      optional_fields          = []
      schedule_frequency       = "Daily"
      sse_kms_key_id           = null
    }
  }
}
module "s3" {
  source               = "./../modules/s3/provision"
  buckets_inventory    = local.buckets_inventory
  buckets              = local.buckets
  bucket_policies      = local.bucket_policies
  buckets_analytics    = local.buckets_analytics
  bucket_notifications = local.bucket_notifications
}
output "s3_generated_resources" {
  value = module.s3.generated_resources
}
output "s3_analytics_resources" {
  value = module.s3.analytics_resources
}
output "s3_analytics_resources_list" {
  value = module.s3.analytics_resources_list
}
output "s3_analytics_resources_map" {
  value = module.s3.analytics_resources_map
}

Debug Output

https://gist.github.com/emagutu/062e0cd97b21c7913d11196a3a32c551

Crash Output

https://gist.github.com/emagutu/caa045280e44f9fb8e898fef9af73b16

Expected Behavior

s3 bucket inventory should be configured.

Actual Behavior

terraform crashed

Steps to Reproduce

1. terraform init
2. terraform apply

Additional Context

The terraform plan is successful but the terraform apply fails.

References

n/a

[bflad]

Hi @emagutu 👋 Thank you for filing this and sorry you ran into trouble here. There is a code issue in the aws_s3_bucket_inventory resource here, which will need to be addressed. You can likely also workaround this in your configuration by ensuring that the filter block is not present at all instead of empty (filter {}) -- if you are using dynamic blocks, this might be done via something along the lines of:

resource "aws_s3_bucket_inventory" "example" {
  # ... other configuration ...

  dynamic "filter" {
    for_each = len(var.filters) == 0 : [] : var.filters
    content {
      # ... configuration ...
    } 
  }
}

For more further help with your specific configuration, it may be best to reach out to the HashiCorp Community Forums where there are far more people ready to help, whereas the GitHub issues here are generally monitored only by the small set of code maintainers and as such we are not the best resource for tailored workarounds.

Relevant portion of crash log:

2021-01-04T08:28:18.107+0300 [DEBUG] plugin.terraform-provider-aws_v3.22.0_x5: panic: interface conversion: interface {} is nil, not map[string]interface {}
2021-01-04T08:28:18.107+0300 [DEBUG] plugin.terraform-provider-aws_v3.22.0_x5: 
2021-01-04T08:28:18.107+0300 [DEBUG] plugin.terraform-provider-aws_v3.22.0_x5: goroutine 86 [running]:
2021-01-04T08:28:18.107+0300 [DEBUG] plugin.terraform-provider-aws_v3.22.0_x5: github.com/terraform-providers/terraform-provider-aws/aws.resourceAwsS3BucketInventoryPut(0xc0010b8100, 0x66e94c0, 0xc0017cf600, 0x1, 0xffffffffffffffff)
2021-01-04T08:28:18.107+0300 [DEBUG] plugin.terraform-provider-aws_v3.22.0_x5: 	/opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/hashicorp/terraform-provider-aws/aws/resource_aws_s3_bucket_inventory.go:213 +0xcdf
2021-01-04T08:28:18.107+0300 [DEBUG] plugin.terraform-provider-aws_v3.22.0_x5: github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).create(0xc0009ad1e0, 0x7e27d80, 0xc000134640, 0xc0010b8100, 0x66e94c0, 0xc0017cf600, 0x0, 0x0, 0x0)
2021-01-04T08:28:18.107+0300 [DEBUG] plugin.terraform-provider-aws_v3.22.0_x5: 	/opt/teamcity-agent/work/5d79fe75d4460a2f/pkg/mod/github.com/hashicorp/terraform-plugin-sdk/v2@v2.3.0/helper/schema/resource.go:268 +0x88

Code:

terraform-provider-aws/aws/resource_aws_s3_bucket_inventory.go

Lines 211 to 213 in 50a3934

	if v, ok := d.GetOk("filter"); ok {
	filterList := v.([]interface{})
	filterMap := filterList[0].(map[string]interface{})

We have very recently introduced guidance on this in the contributing guide, but we need to go through the existing code to find and fix places this issue exists still.

[emagutu]

Hi @bflad,
The workaround worked. Thanks a lot!

[bflad]

The fix for this has been merged and will release with version 3.24.0 of the Terraform AWS Provider, later this week. 👍

[ghost]

This has been released in version 3.24.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!