Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(aws-s3): log delivery may be incorrectly configured when target bucket is imported #23552

Merged
merged 5 commits into from
Jan 12, 2023

Conversation

kylelaker
Copy link
Contributor

@kylelaker kylelaker commented Jan 4, 2023

This resolves issues with log delivery for situations where the target bucket for S3 Access Logs is in another stack. This situation was previously missing from unit and integration tests. This adds additional checks to make sure that this behavior works. There are two primary issues here:

  1. Regardless of @aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy, if the target bucket was imported (not a concrete Bucket), the grant would be applied to the source bucket for logs improperly. This resulted in the ACL or Bucket Policy being added to the wrong stack.
  2. When @aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy was enabled the created bucket policy resulted in a cyclical dependency because of the conditions; these conditions are not necessary for successful delivery. This omits them unless the bucket is concrete and in the same stack.

Unit tests and integration tests now cover importing a bucket between stacks.

Closes: #23547
Closes: #23588


All Submissions:

Adding new Construct Runtime Dependencies:

  • This PR adds new construct runtime dependencies following the process described here

New Features

  • Have you added the new feature to an integration test?
    • Did you use yarn integ to deploy the infrastructure and generate the snapshot (i.e. yarn integ without --dry-run)?

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@gitpod-io
Copy link

gitpod-io bot commented Jan 4, 2023

@aws-cdk-automation aws-cdk-automation requested a review from a team January 4, 2023 00:54
@github-actions github-actions bot added admired-contributor [Pilot] contributed between 13-24 PRs to the CDK p2 labels Jan 4, 2023
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

@kylelaker kylelaker marked this pull request as ready for review January 4, 2023 01:28
@github-actions github-actions bot added bug This issue is a bug. effort/small Small work item – less than a day of effort labels Jan 4, 2023
@kylelaker kylelaker changed the title fix(aws-s3): log delivery may be incorrectly configured for imported buckets fix(aws-s3): log delivery may be incorrectly configured when target bucket is imported Jan 4, 2023
@kylelaker
Copy link
Contributor Author

kylelaker commented Jan 4, 2023

Is there an effective way to add an integration test for this case? We need to check that an imported resource (via fromXxx) builds. But that will require a static resource name. Happy to follow an example (or this branch also allows maintainer edits)

Edit: Done

@aws-cdk-automation aws-cdk-automation dismissed their stale review January 7, 2023 16:26

✅ Updated pull request passes all PRLinter validations. Dissmissing previous PRLinter review.

Naumel
Naumel previously approved these changes Jan 12, 2023
Copy link
Contributor

@Naumel Naumel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Between the references, explanation and test coverage, this has made my day. Nice work!

@mergify
Copy link
Contributor

mergify bot commented Jan 12, 2023

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@Naumel
Copy link
Contributor

Naumel commented Jan 12, 2023

@Mergifyio update

@mergify
Copy link
Contributor

mergify bot commented Jan 12, 2023

update

❌ Base branch update has failed

refusing to allow a GitHub App to create or update workflow .github/workflows/issue-label-assign.yml without workflows permission
err-code: D04ED

@mergify mergify bot dismissed Naumel’s stale review January 12, 2023 11:43

Pull request has been modified.

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: 3cf98df
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify
Copy link
Contributor

mergify bot commented Jan 12, 2023

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot merged commit 41327d8 into aws:main Jan 12, 2023
@lkr2des
Copy link

lkr2des commented Jan 12, 2023

I am waiting for this fix release.Looks like this fix didn't make it to v2.60.0 [https://github.com/aws/aws-cdk/releases/tag/v2.60.0].Any idea which release of CDK version this fix would be in?.

@robertd
Copy link
Contributor

robertd commented Jan 12, 2023

It was merged in master... so probably in the next one v2.61.0

@robertd
Copy link
Contributor

robertd commented Jan 12, 2023

I wonder if release automation didn't pick it up because it's not following the convention fix(aws-s3) vs fix(s3).

@robertd
Copy link
Contributor

robertd commented Jan 12, 2023

I take that back... it literally happened 9hrs after they cut the v2.60.0 release. However... I hope it ends up in the release changelog section (see my previous comment on following the naming convention).

In any case, @kylelaker thanks for fixing this. 👍

@kylelaker
Copy link
Contributor Author

I hope it ends up in the release changelog section (see my previous comment on following the naming convention

Ahh rats! I totally missed getting the convention wrong (too used to using aws-s3 in code I guess 😅). I took a look at the history and it looks like for some reason v2.54.0 had a few commits that incorrectly had aws- in the scope (and v2.56.0 had two and v2.260.0 had one). It looks like the changelog script just extracts the scope and dumps it into the changelog (so it won't be normalized to s3, it'll just remain aws-s3).

This might be a decent addition to prlint to nudge towards a little more consistency.

kylelaker added a commit to kylelaker/aws-cdk that referenced this pull request Jan 13, 2023
The AWS CDK uses conventional commits which encourage a scope. In nearly
all cases, contributors omit the `aws-` prefix; though there are some
situations where they're included (such as aws#23552, aws#23225, and others).
Interestingly, it seems that most often the `aws-` prefix is used for
chore/doc commits that don't end up in changelogs anyway.

This actually results in inconsistencies for users in the changelog.
Because the changelog sorts entries alphabetically by scope, changes
that were contributed with `aws-s3` as the scope are listed at the top
of the changelog whereas changes that just used `s3` are sorted further
down. This means it's harder for users to review one (or I suppose 2
with Feature/Fix being separate sections) spots in the changelog to
identify specific modules they care about.

It's also possible that rather than potentially annoying contributors
who have to edit their titles, this should be automatically fixed up in
the changelog generation. The current behavior may also accidentally
encourage users to scan the whole changelog rather than just a few bits
of it.

So it's very understandable if the potential inconvenience from this
change it's worth the minor formatting consistency gain.
kylelaker added a commit to kylelaker/aws-cdk that referenced this pull request Jan 13, 2023
The AWS CDK uses conventional commits which encourage a scope. In nearly
all cases, contributors omit the `aws-` prefix; though there are some
situations where they're included (such as aws#23552, aws#23225, and others).
Interestingly, it seems that most often the `aws-` prefix is used for
chore/doc commits that don't end up in changelogs anyway.

This actually results in inconsistencies for users in the changelog.
Because the changelog sorts entries alphabetically by scope, changes
that were contributed with `aws-s3` as the scope are listed at the top
of the changelog whereas changes that just used `s3` are sorted further
down. This means it's harder for users to review one (or I suppose 2
with Feature/Fix being separate sections) spots in the changelog to
identify specific modules they care about.

It's also possible that rather than potentially annoying contributors
who have to edit their titles, this should be automatically fixed up in
the changelog generation. The current behavior may also accidentally
encourage users to scan the whole changelog rather than just a few bits
of it.

So it's very understandable if the potential inconvenience from this
change it's worth the minor formatting consistency gain.
mergify bot pushed a commit that referenced this pull request Jan 14, 2023
The AWS CDK uses conventional commits which encourage a scope. In nearly
all cases, contributors omit the `aws-` prefix; though there are some
situations where they're included (such as #23552, #23225, and others).
Interestingly, it seems that most often the `aws-` prefix is used for
chore/doc commits that don't end up in changelogs anyway.

This actually results in inconsistencies for users in the changelog.
Because the changelog sorts entries alphabetically by scope, changes
that were contributed with `aws-s3` as the scope are listed at the top
of the changelog whereas changes that just used `s3` are sorted further
down. This means it's harder for users to review one (or I suppose 2
with Feature/Fix being separate sections) spots in the changelog to
identify specific modules they care about.

It's also possible that rather than potentially annoying contributors
who have to edit their titles, this should be automatically fixed up in
the changelog generation. The current behavior may also accidentally
encourage users to scan the whole changelog rather than just a few bits
of it.

So it's very understandable if the potential inconvenience from this
change it's worth the minor formatting consistency gain.


----

### All Submissions:

* [X] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md)

### Adding new Construct Runtime Dependencies:

* [ ] This PR adds new construct runtime dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md/#adding-construct-runtime-dependencies)

### New Features

* [ ] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/main/INTEGRATION_TESTS.md)?
	* [ ] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@lkr2des
Copy link

lkr2des commented Jan 20, 2023

Thanks @kylelaker that fix made it to v2.61.0, which resolved the s3 issue noticed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
admired-contributor [Pilot] contributed between 13-24 PRs to the CDK bug This issue is a bug. effort/small Small work item – less than a day of effort p2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

(s3): CDK sets ACL on all buckets cdk: Breaking change to S3 bucket server access logging (v2.59.0)
5 participants