Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(aws-s3): create default bucket policy when required (under feature flag) #20765

Merged
merged 6 commits into from
Jul 6, 2022

Conversation

corymhall
Copy link
Contributor

@corymhall corymhall commented Jun 16, 2022

Created a new feature flag @aws-cdk/aws-s3:createDefaultLoggingPolicy

There are certain types of S3 Buckets that AWS will automatically create a bucket policy for you if you do not create one. For example, if you create an S3 Bucket to be used as the destination for VPC Flow Logs and you do not create a Bucket Policy, AWS will automatically create a bucket policy for you. The full list of resources can be found here

  • Vpc Flow Logs
    - [ ] AWS Network Firewall logs (No L2 support yet)
    - [ ] AWS Global Accelerator flow logs (not currently possible]
    - [ ] EC2 Spot Instance data feed(no cloudformation support yet)
    - [ ] CloudFront access logs & streaming access logs (CloudFront uses bucket ACL not bucket policy)
  • Network Load Balancer access logs (already done)
  • Amazon Managed Streaming for Apache Kafka broker logs

If we allow AWS to create these policies automatically, it prevents CDK from every managing that policy in the future. Since we know what the policy should be we should instead create the logging bucket with the required policy.

fixes #18816


All Submissions:

Adding new Unconventional Dependencies:

  • This PR adds new unconventional dependencies following the process described here

New Features

  • Have you added the new feature to an integration test?
    • Did you use yarn integ to deploy the infrastructure and generate the snapshot (i.e. yarn integ without --dry-run)?

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

…re flag)

Created a new feature flag `@aws-cdk/aws-s3:defaultBucketPolicy`

Working on adding the policies for the following cases:

- [X] Vpc Flow Logs
- [ ] AWS Network Firewall logs
- [ ] AWS Global Accelerator flow logs
- [ ] EC2 Spot Instance data feed
- [ ] CloudFront access logs & streaming access logs
- [ ] Network Load Balancer access logs
- [ ] Amazon Managed Streaming for Apache Kafka broker logs

fixes #18816
@gitpod-io
Copy link

gitpod-io bot commented Jun 16, 2022

@corymhall corymhall changed the title feat(aws-s3): create default bucket policy when required (under featu… feat(aws-s3): create default bucket policy when required (under feature flag) Jun 16, 2022
@aws-cdk-automation aws-cdk-automation requested a review from a team June 16, 2022 16:10
@github-actions github-actions bot added effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p1 labels Jun 16, 2022
@mergify mergify bot added the contribution/core This is a PR that came from AWS. label Jun 16, 2022
@corymhall corymhall marked this pull request as ready for review June 29, 2022 14:32
@mergify
Copy link
Contributor

mergify bot commented Jul 6, 2022

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot merged commit cefa453 into aws:main Jul 6, 2022
@mergify
Copy link
Contributor

mergify bot commented Jul 6, 2022

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: a9727a6
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

iliapolo pushed a commit that referenced this pull request Jul 6, 2022
…re flag) (#20765)

Created a new feature flag `@aws-cdk/aws-s3:createDefaultLoggingPolicy`

There are certain types of S3 Buckets that AWS will automatically create a bucket policy for you if you do not create one. For example, if you create an S3 Bucket to be used as the destination for VPC Flow Logs and you  do not create a Bucket Policy, AWS will automatically create a bucket policy for you. The full list of resources can be found [here](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AWS-logs-and-resource-policy.html#AWS-logs-infrastructure-S3)

- [X] Vpc Flow Logs
~- [ ] AWS Network Firewall logs~ (No L2 support yet)
~- [ ] AWS Global Accelerator flow logs~ ([not currently possible](aws-cloudformation/cloudformation-coverage-roadmap#922)]
~- [ ] EC2 Spot Instance data feed~(no cloudformation support yet)
~- [ ] CloudFront access logs & streaming access logs~ (CloudFront uses bucket ACL _not_ bucket policy)
- [X] Network Load Balancer access logs (already done)
- [x] Amazon Managed Streaming for Apache Kafka broker logs

If we allow AWS to create these policies automatically, it prevents CDK from every managing that policy in the future. Since we know what the policy should be we should instead create the logging bucket with the required policy.

fixes #18816


----

### All Submissions:

* [x] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [x] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/main/INTEGRATION_TESTS.md)?
	* [x] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
mergify bot pushed a commit that referenced this pull request Jul 7, 2022
…1042)

PR #20765 introduced destinationOptions, but only introduced one of the
optional properties ('hiveCompatiblePartitions') since that is the only
property that was relevant for the PR. The [docs](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-flowlog.html#cfn-ec2-flowlog-destinationoptions)
don't specify this, but if you provide `destinationOptions` you must
specify a value for each prop, otherwise you will receive an error
message on deploy.

This PR adds the two additional properties.

re #21037


----

### All Submissions:

* [ ] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [ ] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/main/INTEGRATION_TESTS.md)?
	* [ ] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
daschaa pushed a commit to daschaa/aws-cdk that referenced this pull request Jul 9, 2022
…re flag) (aws#20765)

Created a new feature flag `@aws-cdk/aws-s3:createDefaultLoggingPolicy`

There are certain types of S3 Buckets that AWS will automatically create a bucket policy for you if you do not create one. For example, if you create an S3 Bucket to be used as the destination for VPC Flow Logs and you  do not create a Bucket Policy, AWS will automatically create a bucket policy for you. The full list of resources can be found [here](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AWS-logs-and-resource-policy.html#AWS-logs-infrastructure-S3)

- [X] Vpc Flow Logs
~- [ ] AWS Network Firewall logs~ (No L2 support yet)
~- [ ] AWS Global Accelerator flow logs~ ([not currently possible](aws-cloudformation/cloudformation-coverage-roadmap#922)]
~- [ ] EC2 Spot Instance data feed~(no cloudformation support yet)
~- [ ] CloudFront access logs & streaming access logs~ (CloudFront uses bucket ACL _not_ bucket policy)
- [X] Network Load Balancer access logs (already done)
- [x] Amazon Managed Streaming for Apache Kafka broker logs

If we allow AWS to create these policies automatically, it prevents CDK from every managing that policy in the future. Since we know what the policy should be we should instead create the logging bucket with the required policy.

fixes aws#18816


----

### All Submissions:

* [x] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [x] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/main/INTEGRATION_TESTS.md)?
	* [x] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
daschaa pushed a commit to daschaa/aws-cdk that referenced this pull request Jul 9, 2022
…s#21042)

PR aws#20765 introduced destinationOptions, but only introduced one of the
optional properties ('hiveCompatiblePartitions') since that is the only
property that was relevant for the PR. The [docs](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-flowlog.html#cfn-ec2-flowlog-destinationoptions)
don't specify this, but if you provide `destinationOptions` you must
specify a value for each prop, otherwise you will receive an error
message on deploy.

This PR adds the two additional properties.

re aws#21037


----

### All Submissions:

* [ ] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [ ] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/main/INTEGRATION_TESTS.md)?
	* [ ] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contribution/core This is a PR that came from AWS. effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

s3 bucket policy resource should be created in cases where AWS will create one automatically
3 participants