Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(s3): S3_002: check bucket has no replication to other region #87

Merged
merged 4 commits into from
Mar 27, 2023

Conversation

corrieriluca
Copy link
Member

@corrieriluca corrieriluca commented Mar 24, 2023

This fixes #5

Replacing the AWS_S3_002 check

This PR replaces the check AWS_S3_002 "S3 buckets are not global but in one zone" because it was inaccurate (see #5).

The new check is "S3 buckets are not replicated to another region". It checks the bucket replication configuration if at least one replication rule has a destination bucket in another region than the current bucket.

Test

Context:

  • Single AWS account, spanning over 2 regions (us-east-1 and eu-west-1)
  • 2 config entries in .yatas.yml, one per region :
    - pluginName: aws
      accounts:
        - profile: ""
          sso: false
          region: "eu-west-1"
          name: "eu-west-1 account"
        - profile: ""
          sso: false
          region: "us-east-1"
          name: "us-east-1 account"

Setup:

  • Create bucket "yatas-test-eu-storage" in "eu-west-1"
  • Create bucket "yatas-test-us-storage" in "us-east-1"
  • Configure replication on "yatas-test-eu-storage" to replicate some objects (or all of them) to "yatas-test-us-storage"

Run YATAS on your setup (including only the AWS_S3_002 check):

Name: eu-west-1 account (0/1)
❌ AWS_S3_002 S3 buckets are not replicated to another region - 0/1
        S3 bucket yatas-test-eu-storage is replicated to the us-east-1 region

Name: us-east-1 account (1/1)
✅ AWS_S3_002 S3 buckets are not replicated to another region - 1/1
        S3 bucket yatas-test-us-storage is not replicated in another region

Implementation details

This PR adds the GetS3ToReplicationOtherRegion getter to S3 package which iterates over the bucket of the current region (passed by parameters) and check if a replication configuration is enabled on the bucket and if so, if a replication rule replicates to a bucket which is in another region.

func GetS3ToReplicationOtherRegion(s aws.Config, b []types.Bucket) []S3ToReplicationOtherRegion {

In order for this implementation to work I had to slightly modify the CheckS3Location function to return also the region of the bucket that is compared.

I also fixed a bug that was unknown to this day: if the bucket was in the us-east-1 region, the function was always returning false. This is because, for the us-east-1 region, the AWS API returns a LocationConstraint of null (empty string in Go). A new check is therefore implemented:

yatas-aws/aws/s3/s3.go

Lines 26 to 36 in 4fcedd4

if string(resp.LocationConstraint) == region {
return true, region
} else if string(resp.LocationConstraint) == "" { // If the bucket is in us-east-1, the LocationConstraint is empty
if region == "us-east-1" {
return true, region
} else {
return false, "us-east-1"
}
} else {
return false, string(resp.LocationConstraint)
}

@corrieriluca corrieriluca marked this pull request as ready for review March 26, 2023 16:04
Copy link
Contributor

@StanGirard StanGirard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Top ! Pas mal le petit smithy pour les erreurs.

@StanGirard StanGirard merged commit fa8ff6c into main Mar 27, 2023
@corrieriluca corrieriluca deleted the feat/s3-buckets-replication-check branch March 27, 2023 08:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

AWS_S3_002 Check is not accurate
2 participants