Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

importing an S3 bucket fails on "Error importing AWS S3 bucket policy" #9508

Closed
ysgard opened this issue Jul 25, 2019 · 5 comments · Fixed by #14121
Closed

importing an S3 bucket fails on "Error importing AWS S3 bucket policy" #9508

ysgard opened this issue Jul 25, 2019 · 5 comments · Fixed by #14121
Assignees
Labels
bug Addresses a defect in current functionality. service/s3 Issues and PRs that pertain to the s3 service.
Milestone

Comments

@ysgard
Copy link

ysgard commented Jul 25, 2019

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

v0.12.5

Affected Resource(s)

  • aws_s3_bucket

Terraform Configuration Files

resource "aws_s3_bucket" "xxxx-data-intelligence" {
  bucket = "xxxx-data-intelligence"
  acl = "private"
}

I have not defined a policy in the configuration, and there is no policy attached to the bucket.

Note that this bucket is in a subaccount than the one we typically use. To handle this, we map our provider account id to terraform workspace, like so:

provider "aws" {
  assume_role {
    role_arn = "arn:aws:iam::${lookup(module.accounts.ids, terraform.workspace, module.accounts.ids["di"])}:role/OrganizationAccountAccessRole"
  }
  region = "us-west-2"
}

This has been working for us for a variety of configurations, this is the first time we've encountered this issue.

Debug Output

➜  common git:(master) ✗ terraform import aws_s3_bucket.xxxx_data_intelligence xxxx-data-intelligence
aws_s3_bucket.xxxx_data_intelligence: Importing from ID "xxxx-data-intelligence"...

Error: Error importing AWS S3 bucket policy: AccessDenied: Access Denied
        status code: 403, request id: E1FAE91EE442xxxx, host id: <snip>

Expected Behavior

The S3 bucket should have been imported into the state.

Actual Behavior

The bucket failed to import, producing the error message mentioned in the debug output.

I suspect what's happening is that the aws provider is correctly assuming role to import the bucket, but doesn't assume role to get the policy, resulting in the 'AccessDenied'.

Steps to Reproduce

  1. Create a provider configuration that defines an AWS account to assume using assume_role.
  2. Create an S3 bucket in that account
  3. Create a terraform configuration that maps to that bucket.
  4. Import the bucket using terraform import

Important Factoids

We're using a provider with assume_role to import the resource from a subaccount.

@github-actions github-actions bot added the needs-triage Waiting for first response or review from a maintainer. label Jul 25, 2019
@aeschright aeschright added the service/s3 Issues and PRs that pertain to the s3 service. label Aug 2, 2019
@ryndaniels ryndaniels added bug Addresses a defect in current functionality. and removed needs-triage Waiting for first response or review from a maintainer. labels Nov 28, 2019
@tbondarchuk
Copy link

I've just encountered the same issue on terraform v0.12.18, aws provider v92.43.0, but was able to import successfully after replacing variables in provider configuration by static values (basically hardcoded account id just for import). I suspect it might be related to how terraform is resolving variables during import, something like hashicorp/terraform#21662

@bflad bflad added this to the v3.0.0 milestone Feb 5, 2020
@karthikeayan
Copy link

Ran into this.

[karthik@server]$ terraform version
Terraform v0.12.24
+ provider.aws v2.61.0

@bflad bflad self-assigned this Jul 9, 2020
bflad added a commit that referenced this issue Jul 9, 2020
Reference: #394
Reference: #9001
Reference: #9508
Reference: #12805

Output from acceptance testing:

```
--- PASS: TestAccAWSS3Bucket_acceleration (70.53s)
--- PASS: TestAccAWSS3Bucket_AclToGrant (64.37s)
--- PASS: TestAccAWSS3Bucket_basic (37.90s)
--- PASS: TestAccAWSS3Bucket_Bucket_EmptyString (39.08s)
--- PASS: TestAccAWSS3Bucket_Cors_Delete (32.28s)
--- PASS: TestAccAWSS3Bucket_Cors_EmptyOrigin (39.25s)
--- PASS: TestAccAWSS3Bucket_Cors_Update (68.80s)
--- PASS: TestAccAWSS3Bucket_disableDefaultEncryption_whenDefaultEncryptionIsEnabled (67.23s)
--- PASS: TestAccAWSS3Bucket_enableDefaultEncryption_whenAES256IsUsed (37.19s)
--- PASS: TestAccAWSS3Bucket_enableDefaultEncryption_whenTypical (44.32s)
--- PASS: TestAccAWSS3Bucket_forceDestroy (37.21s)
--- PASS: TestAccAWSS3Bucket_forceDestroyWithEmptyPrefixes (38.50s)
--- PASS: TestAccAWSS3Bucket_forceDestroyWithObjectLockEnabled (37.77s)
--- PASS: TestAccAWSS3Bucket_generatedName (38.80s)
--- PASS: TestAccAWSS3Bucket_GrantToAcl (60.31s)
--- PASS: TestAccAWSS3Bucket_LifecycleBasic (89.67s)
--- PASS: TestAccAWSS3Bucket_LifecycleExpireMarkerOnly (67.52s)
--- PASS: TestAccAWSS3Bucket_LifecycleRule_Expiration_EmptyConfigurationBlock (30.08s)
--- PASS: TestAccAWSS3Bucket_Logging (56.73s)
--- PASS: TestAccAWSS3Bucket_namePrefix (40.92s)
--- PASS: TestAccAWSS3Bucket_objectLock (68.34s)
--- PASS: TestAccAWSS3Bucket_Policy (97.07s)
--- PASS: TestAccAWSS3Bucket_region (34.45s)
--- PASS: TestAccAWSS3Bucket_Replication (159.22s)
--- PASS: TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AccessControlTranslation (94.18s)
--- PASS: TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AddAccessControlTranslation (95.79s)
--- PASS: TestAccAWSS3Bucket_ReplicationExpectVersioningValidationError (28.62s)
--- PASS: TestAccAWSS3Bucket_ReplicationSchemaV2 (167.50s)
--- PASS: TestAccAWSS3Bucket_ReplicationWithoutPrefix (55.52s)
--- PASS: TestAccAWSS3Bucket_ReplicationWithoutStorageClass (58.02s)
--- PASS: TestAccAWSS3Bucket_RequestPayer (67.28s)
--- PASS: TestAccAWSS3Bucket_shouldFailNotFound (19.65s)
--- PASS: TestAccAWSS3Bucket_tagsWithNoSystemTags (119.32s)
--- PASS: TestAccAWSS3Bucket_tagsWithSystemTags (171.42s)
--- PASS: TestAccAWSS3Bucket_UpdateAcl (65.51s)
--- PASS: TestAccAWSS3Bucket_UpdateGrant (92.38s)
--- PASS: TestAccAWSS3Bucket_Versioning (95.55s)
--- PASS: TestAccAWSS3Bucket_Website_Simple (95.12s)
--- PASS: TestAccAWSS3Bucket_WebsiteRedirect (91.21s)
--- PASS: TestAccAWSS3Bucket_WebsiteRoutingRules (65.48s)
```
bflad added a commit that referenced this issue Jul 13, 2020
…14121)

Reference: #394
Reference: #9001
Reference: #9508
Reference: #12805

Output from acceptance testing:

```
--- PASS: TestAccAWSS3Bucket_acceleration (70.53s)
--- PASS: TestAccAWSS3Bucket_AclToGrant (64.37s)
--- PASS: TestAccAWSS3Bucket_basic (37.90s)
--- PASS: TestAccAWSS3Bucket_Bucket_EmptyString (39.08s)
--- PASS: TestAccAWSS3Bucket_Cors_Delete (32.28s)
--- PASS: TestAccAWSS3Bucket_Cors_EmptyOrigin (39.25s)
--- PASS: TestAccAWSS3Bucket_Cors_Update (68.80s)
--- PASS: TestAccAWSS3Bucket_disableDefaultEncryption_whenDefaultEncryptionIsEnabled (67.23s)
--- PASS: TestAccAWSS3Bucket_enableDefaultEncryption_whenAES256IsUsed (37.19s)
--- PASS: TestAccAWSS3Bucket_enableDefaultEncryption_whenTypical (44.32s)
--- PASS: TestAccAWSS3Bucket_forceDestroy (37.21s)
--- PASS: TestAccAWSS3Bucket_forceDestroyWithEmptyPrefixes (38.50s)
--- PASS: TestAccAWSS3Bucket_forceDestroyWithObjectLockEnabled (37.77s)
--- PASS: TestAccAWSS3Bucket_generatedName (38.80s)
--- PASS: TestAccAWSS3Bucket_GrantToAcl (60.31s)
--- PASS: TestAccAWSS3Bucket_LifecycleBasic (89.67s)
--- PASS: TestAccAWSS3Bucket_LifecycleExpireMarkerOnly (67.52s)
--- PASS: TestAccAWSS3Bucket_LifecycleRule_Expiration_EmptyConfigurationBlock (30.08s)
--- PASS: TestAccAWSS3Bucket_Logging (56.73s)
--- PASS: TestAccAWSS3Bucket_namePrefix (40.92s)
--- PASS: TestAccAWSS3Bucket_objectLock (68.34s)
--- PASS: TestAccAWSS3Bucket_Policy (97.07s)
--- PASS: TestAccAWSS3Bucket_region (34.45s)
--- PASS: TestAccAWSS3Bucket_Replication (159.22s)
--- PASS: TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AccessControlTranslation (94.18s)
--- PASS: TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AddAccessControlTranslation (95.79s)
--- PASS: TestAccAWSS3Bucket_ReplicationExpectVersioningValidationError (28.62s)
--- PASS: TestAccAWSS3Bucket_ReplicationSchemaV2 (167.50s)
--- PASS: TestAccAWSS3Bucket_ReplicationWithoutPrefix (55.52s)
--- PASS: TestAccAWSS3Bucket_ReplicationWithoutStorageClass (58.02s)
--- PASS: TestAccAWSS3Bucket_RequestPayer (67.28s)
--- PASS: TestAccAWSS3Bucket_shouldFailNotFound (19.65s)
--- PASS: TestAccAWSS3Bucket_tagsWithNoSystemTags (119.32s)
--- PASS: TestAccAWSS3Bucket_tagsWithSystemTags (171.42s)
--- PASS: TestAccAWSS3Bucket_UpdateAcl (65.51s)
--- PASS: TestAccAWSS3Bucket_UpdateGrant (92.38s)
--- PASS: TestAccAWSS3Bucket_Versioning (95.55s)
--- PASS: TestAccAWSS3Bucket_Website_Simple (95.12s)
--- PASS: TestAccAWSS3Bucket_WebsiteRedirect (91.21s)
--- PASS: TestAccAWSS3Bucket_WebsiteRoutingRules (65.48s)
```
@bflad
Copy link
Member

bflad commented Jul 13, 2020

The removal of the automatic aws_s3_bucket_policy resource import during aws_s3_bucket resource import has been merged and will release with version 3.0.0 of the Terraform AWS Provider, likely in two weeks. Please follow the v3.0.0 milestone for tracking the progress of that release. You can use the aws_s3_bucket_policy resource import support to import that resource directly after the provider upgrade, similar to all other Terraform resources that support import. 👍

@ghost
Copy link

ghost commented Jul 31, 2020

This has been released in version 3.0.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

@ghost
Copy link

ghost commented Aug 13, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Aug 13, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Addresses a defect in current functionality. service/s3 Issues and PRs that pertain to the s3 service.
Projects
None yet
6 participants