provider/aws: Add support for CloudTrail log validation + KMS encryption

[radeksimko]

Acceptance tests

$ make testacc TEST=./builtin/providers/aws TESTARGS='-run=AWSCloudTrail'

==> Checking that code complies with gofmt requirements...
/Users/rsimko1016/gopath/bin/stringer
GO15VENDOREXPERIMENT=1 go generate $(GO15VENDOREXPERIMENT=1 go list ./... | grep -v /vendor/)
TF_ACC=1 GO15VENDOREXPERIMENT=1 go test ./builtin/providers/aws -v -run=AWSCloudTrail -timeout 120m
=== RUN   TestAccAWSCloudTrail_basic
--- PASS: TestAccAWSCloudTrail_basic (34.97s)
=== RUN   TestAccAWSCloudTrail_enable_logging
--- PASS: TestAccAWSCloudTrail_enable_logging (47.99s)
=== RUN   TestAccAWSCloudTrail_is_multi_region
--- PASS: TestAccAWSCloudTrail_is_multi_region (41.89s)
=== RUN   TestAccAWSCloudTrail_logValidation
--- PASS: TestAccAWSCloudTrail_logValidation (36.64s)
PASS
ok      github.com/hashicorp/terraform/builtin/providers/aws    161.513s

[stack72]

This looks good @radeksimko :)

[stack72]

@radeksimko on the second run of these tests I got the following:

make testacc TEST=./builtin/providers/aws TESTARGS='-run=AWSCloudTrail'
==> Checking that code complies with gofmt requirements...
/Users/stacko/Code/go/bin/stringer
GO15VENDOREXPERIMENT=1 go generate $(GO15VENDOREXPERIMENT=1 go list ./... | grep -v /vendor/)
TF_ACC=1 GO15VENDOREXPERIMENT=1 go test ./builtin/providers/aws -v -run=AWSCloudTrail -timeout 120m
=== RUN   TestAccAWSCloudTrail_basic
--- PASS: TestAccAWSCloudTrail_basic (37.34s)
=== RUN   TestAccAWSCloudTrail_enable_logging
--- PASS: TestAccAWSCloudTrail_enable_logging (51.31s)
=== RUN   TestAccAWSCloudTrail_is_multi_region
--- FAIL: TestAccAWSCloudTrail_is_multi_region (12.70s)
    testing.go:148: Step 0 error: Error applying: 1 error(s) occurred:

        * aws_cloudtrail.foobar: InsufficientS3BucketPolicyException: Incorrect S3 bucket policy is detected for bucket: tf-test-trail-1364698938312292751
            status code: 400, request id: e7d436d3-ce78-11e5-b0af-73282d2d776d
=== RUN   TestAccAWSCloudTrail_logValidation
--- PASS: TestAccAWSCloudTrail_logValidation (34.87s)
FAIL
exit status 1
FAIL    github.com/hashicorp/terraform/builtin/providers/aws    136.245s

Is there a specific setup i need in my environment to ensure these pass constantly?

[radeksimko]

Hmm, I assume this is going to be caused by IAM eventual consistency?
I could add a Retry block with InsufficientS3BucketPolicyException.

[radeksimko]

That will also eventually make people who really used insufficient bucket policy wait though.

I wish there was a different error code for these... 😢

[jen20]

Right - or some kind of endpoint to poll for IAM operations being "complete" :(

[radeksimko]

I agree this needs solving, in fact I opened #4447 a while back to address such problems, but I don't think it should be solved in the context of this PR.

It's something that affects CloudTrail generally.

I can send a separate PR to work around this policy problem.

Do you guys agree - i.e. can we merge this? @stack72 @jen20

[stack72]

I think we can merge this one and then solve the evential consistency in the next one

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.