r/cloudtrail: Exclude Management Event Sources

[StephenKing]

Community Note

• Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Description

ExcludeManagementEventSources allows to disable KMS events to be included in CloudTrail trails. We paid >$2000 last month to store KMS decrypt events and Terraform doesn't yet support to disable this.

This PR is based on previous work by @DrFaust92 in #12390

Closes #11448

Release note for CHANGELOG:

resource/aws_cloudtrail: Add `exclude_management_event_sources` attribute to `event_selector` configuration block

Affected Resource(s)

• resource: aws_cloudtrail

Terraform Configuration

resource "aws_cloudtrail" "this" {
  name = "test"
  s3_bucket_name = "test"

  event_selector {
    exclude_management_event_sources = ["kms.amazonaws.com"]
  }
}

Output from acceptance testing:

$ make testacc TESTARGS='-run=TestAccAWSCloudTrail/Trail/eventSelectorExclude'      
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSCloudTrail/Trail/eventSelectorExclude -timeout 120m
=== RUN   TestAccAWSCloudTrailServiceAccount_basic
=== PAUSE TestAccAWSCloudTrailServiceAccount_basic
=== RUN   TestAccAWSCloudTrailServiceAccount_Region
=== PAUSE TestAccAWSCloudTrailServiceAccount_Region
=== RUN   TestAccAWSCloudTrail_serial
=== RUN   TestAccAWSCloudTrail_serial/Trail
=== RUN   TestAccAWSCloudTrail_serial/Trail/eventSelectorExclude
--- PASS: TestAccAWSCloudTrail_serial (103.27s)
    --- PASS: TestAccAWSCloudTrail_serial/Trail (103.27s)
        --- PASS: TestAccAWSCloudTrail_serial/Trail/eventSelectorExclude (103.27s)
=== CONT  TestAccAWSCloudTrailServiceAccount_basic
=== CONT  TestAccAWSCloudTrailServiceAccount_Region
--- PASS: TestAccAWSCloudTrailServiceAccount_basic (29.17s)
--- PASS: TestAccAWSCloudTrailServiceAccount_Region (29.83s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       134.596s

[ohmer]

any chance this gets merged?

[jglopez]

FYI: As of at least July 6, CloudTrail now also allows the ability to "Exclude Amazon RDS Data API events" (rdsdata.amazonaws.com) too.

https://docs.aws.amazon.com/awscloudtrail/latest/APIReference/API_EventSelector.html#awscloudtrail-Type-EventSelector-ExcludeManagementEventSources
https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/logging-using-cloudtrail-data-api.html#logging-using-cloudtrail-data-api.excluding-cloudtrail-events

[Zordrak]

The lack of this feature is costing clients lots of money where we are tracking high volumes of events we now have the option to exclude, but can only do so if we move management of cloudtrail to CloudFormation due to the missing support here.

[zhelding]

Pull request #21306 has significantly refactored the AWS Provider codebase. As a result, most PRs opened prior to the refactor now have merge conflicts that must be resolved before proceeding.

Specifically, PR #21306 relocated the code for all AWS resources and data sources from a single aws directory to a large number of separate directories in internal/service, each corresponding to a particular AWS service. This separation of code has also allowed for us to simplify the names of underlying functions -- while still avoiding namespace collisions.

We recognize that many pull requests have been open for some time without yet being addressed by our maintainers. Therefore, we want to make it clear that resolving these conflicts in no way affects the prioritization of a particular pull request. Once a pull request has been prioritized for review, the necessary changes will be made by a maintainer -- either directly or in collaboration with the pull request author.

For a more complete description of this refactor, including examples of how old filepaths and function names correspond to their new counterparts: please refer to issue #20000.

For a quick guide on how to amend your pull request to resolve the merge conflicts resulting from this refactor and bring it in line with our new code patterns: please refer to our Service Package Refactor Pull Request Guide.

[breathingdust]

Hi all 👋 Just letting you know that this is issue is featured on this quarters roadmap. If a PR exists to close the issue a maintainer will review and either make changes directly, or work with the original author to get the contribution merged. If you have written a PR to resolve the issue please ensure the "Allow edits from maintainers" box is checked. Thanks for your patience and we are looking forward to getting this merged soon!

[anGie44]

Thank you for your contribution @StephenKing , overall LGTM 🚀 I've modified the tests a bit to check for each supported exclusion value and added a changelog file to get this into our upcoming release.

Output of acceptance tests (note failure in enableLogging occurs in main, unrelated to these changes):

--- FAIL: TestAccCloudTrail_serial (1015.57s)
    --- FAIL: TestAccCloudTrail_serial/Trail (1015.57s)
        --- PASS: TestAccCloudTrail_serial/Trail/basic (61.42s)
        --- PASS: TestAccCloudTrail_serial/Trail/cloudwatch (72.68s)
        --- PASS: TestAccCloudTrail_serial/Trail/globalServiceEvents (31.89s)
        --- SKIP: TestAccCloudTrail_serial/Trail/organization (0.19s)
        --- PASS: TestAccCloudTrail_serial/Trail/eventSelectorDynamoDB (29.08s)
        --- PASS: TestAccCloudTrail_serial/Trail/multiRegion (82.28s)
        --- PASS: TestAccCloudTrail_serial/Trail/tags (82.45s)
        --- PASS: TestAccCloudTrail_serial/Trail/eventSelector (123.38s)
        --- PASS: TestAccCloudTrail_serial/Trail/logValidation (57.15s)
        --- PASS: TestAccCloudTrail_serial/Trail/advancedEventSelector (34.09s)
        --- PASS: TestAccCloudTrail_serial/Trail/kmsKey (32.56s)
        --- PASS: TestAccCloudTrail_serial/Trail/eventSelectorExclude (329.89s)
        --- PASS: TestAccCloudTrail_serial/Trail/insightSelector (33.44s)
        --- FAIL: TestAccCloudTrail_serial/Trail/enableLogging (45.05s) --- existing failure in main
FAIL
FAIL	github.com/hashicorp/terraform-provider-aws/internal/service/cloudtrail	1019.229s

[github-actions]

This functionality has been released in v3.67.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.