Help with using imported buckets

[lisleyanne20]

Hi,

Been following the blog for the s3Scanner and completed all the steps but when I upload an object, I am getting a Status "Error" on the tag. Here is the cloudwatch metrics:
[ERROR] Exception: {
"source": "serverless-clamscan",
"input_bucket": "bucket",
"input_key": "sample-virus.txt",
"status": "ERROR",
"message": "Forbidden"
}
Traceback (most recent call last):
  File "/var/lang/lib/python3.8/site-packages/aws_lambda_powertools/metrics/metrics.py", line 184, in decorate
    response = lambda_handler(event, context)
  File "/var/lang/lib/python3.8/site-packages/aws_lambda_powertools/logging/logger.py", line 354, in decorate
    return lambda_handler(event, context, *args, **kwargs)
  File "/var/task/lambda.py", line 78, in lambda_handler
    download_object(input_bucket, input_key, payload_path)
  File "/var/task/lambda.py", line 149, in download_object
    report_failure(
  File "/var/task/lambda.py", line 299, in report_failure
    raise Exception(json.dumps(exception_json))

Sample virus was from the blog too. Also asked assistance from AWS Support but we still could not fix the issue. I hope you can help us with this. Thank you.

[dontirun]

If I'm understanding correctly the Lambda function doesn't have permission to download the object, but still has permissions to update the tag.

1. Are you following the example from the blog exactly?
2. Does the account that you are launching the solution in have some account level boundaries? (ex. AWS Organizations Service Control Policies)?

[lisleyanne20]

Hi @dontirun, thank you very much for your response. This means a lot.
As for your question, yes I have followed the blog and the only changes I have was using an already created s3 bucket instead of using a new one.

I am an admin of our AWS account, can you specify what permission should I add for the lambda to download the object I have uploaded in S3? Thank you very much.

[dontirun]

The permission to download the file should already exist on the Lambda function's IAM role and the S3 Bucket's (which you uploaded the file to) Bucket Policy as that is set up automatically by the solution. You can check to confirm that is the case.

Confirming that both those policies were properly set-up could indicate that there is something else preventing the Lambda from downloading the object

[lisleyanne20]

@dontirun I have already added my existing bucket on the lambda role as well as added the bucket policy same with the staging bucket. Also checked all the roles created by the cloudFormation template and I have added the bucket arn to which the bucket permission is needed.

[dontirun]

I don't quite understand the setup here, are you deploying the solution and trying to modify a different bucket to work on the solution after the fact?

[lisleyanne20]

@dontirun sorry for the confusion. Yes, I want to apply the solution to my other buckets. I already added permission on the lambda function for the other buckets. Also added the same bucket policy as the one created on this solution but I still got the clientError.

[dontirun]

Edit: updated example code

---

Okay that makes more sense! That's because the solution creates a S3 Gateway endpoint in the VPC where the scanning function is located and specifically allowlists buckets that can be accessed through the VPC. I would recommend doing the following instead of doing that manually.

1. Make sure you are using the latest version of cdk-serverless-clamscan (currently 2.3.2)
2. Modify the example in the blog to be similar to the following. You can add more buckets if you want, but this should minimally represent what needs to be done. Please Note the warnings in the example

import { Stack, StackProps, App } from 'aws-cdk-lib';
import { Bucket, BucketPolicy } from 'aws-cdk-lib/aws-s3';
import { ServerlessClamscan } from 'cdk-serverless-clamscan';
import { Construct } from 'constructs';

export class CdkTestStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);
    const sc = new ServerlessClamscan(this, 'Clamscan', {
      // When using imported buckets the user is responsible for adding the required policy statement to the bucket policy
      acceptResponsibilityForUsingImportedBucket: true,
    });

    const bucket = Bucket.fromBucketName(this, 'SpecificImportedBucket', 'put-your-imported-bucket-name-here');
    sc.addSourceBucket(bucket);

    // WARNING this will override any existing Bucket Policy on the imported Bucket
    const bucketPolicyForSpecificImportedBucket = new BucketPolicy(this, 'BucketPolicy', {
      bucket: bucket,
    });

    // Generate and add the Resource Policy Statement for the specific imported Bucket
    // You will want to generate a different policy statement for each imported bucket you use
    bucketPolicyForSpecificImportedBucket.document.addStatements(sc.getPolicyStatementForBucket(bucket));
  }
}

[lisleyanne20]

@dontirun Thank you so much! I will try this on my end.

[lisleyanne20]

Hi again @dontirun , when I tried running cdk deploy with the code above, and now I am getting this error:
Missing required field Statement (Service: Amazon S3; Status Code: 400; Error Code: MalformedPolicy; Request

[dontirun]

@lisleyanne20

I made a mistake in my example by adding a policy statement without creating the policy document. That should be fixed now in the edited version.

[lisleyanne20]

@dontirun thank you so much it is working now.
Do you also have a sample for multiple buckets? I tried on my end but it's not working.
This was my code:
import { Stack, StackProps, App } from 'aws-cdk-lib';
import { Bucket, BucketPolicy } from 'aws-cdk-lib/aws-s3';
import { ServerlessClamscan } from 'cdk-serverless-clamscan';
import { Construct } from 'constructs';

export class CdkStack extends Stack {
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props);
const sc = new ServerlessClamscan(this, 'Clamscan', {
// When using imported buckets the user is responsible for adding the required policy statement to the bucket policy
acceptResponsibilityForUsingImportedBucket: true,
});

const bucket = Bucket.fromBucketName(this, 'SpecificImportedBucket', 'bucket1');
sc.addSourceBucket(bucket);

const bucket2 = Bucket.fromBucketName(this, 'SpecificImportedBucket2', 'bucket2');
sc.addSourceBucket(bucket2);

// WARNING this will override any existing Bucket Policy on the imported Bucket
const bucketPolicyForSpecificImportedBucket = new BucketPolicy(this, 'BucketPolicy', {
  bucket: bucket,
});

const bucketPolicyForSpecificImportedBucket2 = new BucketPolicy(this, 'BucketPolicy2', {
  bucket: bucket2,
});
// Generate and add the Resource Policy Statement for the specific imported Bucket
// You will want to generate a different policy statement for each imported bucket you use
bucketPolicyForSpecificImportedBucket.document.addStatements(sc.getPolicyStatementForBucket(bucket));
bucketPolicyForSpecificImportedBucket2.document.addStatements(sc.getPolicyStatementForBucket(bucket2));

}
}

And I'm getting this error:
BucketPolicy2 (BucketPolicy2321325C2) The bucket policy already exists on bucket bucket2.

[dontirun]

The code looks fine. Does bucket2 already have a bucket policy? According to this post you may have to manually delete the existing bucket policy on the bucket before deploying the solution

[lisleyanne20]

The code looks fine. Does bucket2 already have a bucket policy? According to this post you may have to manually delete the existing bucket policy on the bucket before deploying the solution

@dontirun yes, there's an existing policy and I manually removed it and now it is working. Thank you so much for your assistance. I will close this ticket now.

[rickyrich]

This permissions issue may be fixed by setting the ACL to "Bucket owner enforced". Try it and let us know if it worked.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/about-object-ownership.html

[jacobpranay]

Hi,

I tried adding the same for existing bucket but still getting the error

throw new Error('acceptResponsibilityForUsingImportedBucket must be set when adding an imported bucket. When using imported buckets the user is responsible for adding the required policy statement to the bucket policy: getPolicyStatementForBucket() can be used to retrieve the policy statement required by the solution');
^
Error: acceptResponsibilityForUsingImportedBucket must be set when adding an imported bucket. When using imported buckets the user is responsible for adding the required policy statement to the bucket policy: getPolicyStatementForBucket() can be used to retrieve the policy statement required by the solution

Code :

` const sc = new ServerlessClamscan(this, 'Clamscan', {
// When using imported buckets the user is responsible for adding the required policy statement to the bucket policy
acceptResponsibilityForUsingImportedBucket: true,
});

    const bucket = s3.Bucket.fromBucketName(this, 'SpecificImportedBucket', 'virus-test-s3');
    sc.addSourceBucket(bucket);

    // WARNING this will override any existing Bucket Policy on the imported Bucket
    const bucketPolicyForSpecificImportedBucket = new s3.BucketPolicy(this, 'BucketPolicy', {
        bucket: bucket,
    });

    // Generate and add the Resource Policy Statement for the specific imported Bucket
    // You will want to generate a different policy statement for each imported bucket you use
    bucketPolicyForSpecificImportedBucket.document.addStatements(sc.getPolicyStatementForBucket(bucket));`

Version: cdk-serverless-clamscan: "^2.6.208"