CopyObject fails for file starting with "/"

[TommyN]

Copying files starting with "/" using CopyObject fails with the following message:

Message: Amazon.S3.AmazonS3Exception : The specified key does not exist.
---- Amazon.Runtime.Internal.HttpErrorResponseException : Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown.

Expected Behavior

The file should copy correctly.

Note: Uploading the file with PutObjectAsync and reading the file with GetObjectRequest works as expected.

Steps to Reproduce (for bugs)

Here is a simple unit test that fails

        [Fact]
        public async Task TestS3WithFileKeyStartingWithForwardSlash()
        {
            IAmazonS3 s3Client = new AmazonS3Client(RegionEndpoint.EUWest1);
            
            var bucketName = "lambda-filenametest-".ToLower() + DateTime.Now.Ticks;
            var key = "/test.xml";
            var copyKey = "copy of test.xml";
            var content = "This is a test!";

            // Create a bucket and object to setup test data.
            await s3Client.PutBucketAsync(bucketName);
            try
            {
                await s3Client.PutObjectAsync(new PutObjectRequest
                {
                    BucketName = bucketName,
                    Key = key,
                    ContentBody = content
                });

                GetObjectRequest request = new GetObjectRequest
                {
                    BucketName = bucketName,
                    Key = key
                };


                // Read back the file content
                string data;

                using (var response = await s3Client.GetObjectAsync(request))
                using (var responseStream = response.ResponseStream)
                using (var reader = new StreamReader(responseStream))
                {
                    data = reader.ReadToEnd();
                }
                
                Assert.Equal(content, data);

                // Copy file to a new file
                CopyObjectRequest copyRequest = new CopyObjectRequest
                {
                    SourceBucket = bucketName,
                    SourceKey = key,
                    DestinationBucket = bucketName,
                    DestinationKey = copyKey
                };
                var resp = await s3Client.CopyObjectAsync(copyRequest);

                Assert.Equal(HttpStatusCode.OK, resp.HttpStatusCode);

            }
            finally
            {
                // Clean up the test data
                await AmazonS3Util.DeleteS3BucketWithObjectsAsync(s3Client, bucketName);
            }
        }

Context

We're exchanging files with a 3rd party using S3 and for some reason they have prefixed their files with "/". We're writing a Lambda function that will trigger and process each file that is uploaded to the S3 bucket, and a part of the processing is copying the file to another bucket.

Your Environment

Installing AWSSDK.Core 3.3.100.3.
Installing AWSSDK.S3 3.3.101.1.
Installing NuGet package AWSSDK.S3 3.3.101.1.

[klaytaybai]

Thanks for reporting this

[github-actions]

We have noticed this issue has not recieved attention in a year. We will close this issue for now. If you think this is in error, please feel free to comment and reopen the issue.

[TommyN]

I think this should be fixed or closed by someone other than the bot ;)

[mwhouser]

This problem is bigger than just a failure to copy objects starting with "/". There's 3 possible "wrong" things that can happen:

Incorrect Behaviour 1

Command: Copy "/abc" to "/def"
Objects: "/abc" exists, "abc" does not
Behaviour: Exception is thrown (original problem reported)

Incorrect Behaviour 2

Command: Copy "/abc" to "/def"
Objects: "/abc" exists, "abc" also exists
Behaviour: "abc" is copied to "def" (wrong file is copied)

Incorrect Behaviour 3

Command: Copy "/abc" to "/def"
Objects: "/abc" does not exist, "abc" exists
Behaviour: "abc" is copied to "def" (expected to throw exception)

Final Notes

So when the source key starts with a '/' character, then the wrong file will be copied.

When the destination key starts with a '/' character, then the file is copied to the wrong location, possibly destroying existing data in the destination bucket.

Since CopyObject is a tool used for Disaster Recovery, this bug needs to be addressed immediately.

[ashishdhingra]

Closing as duplicate of the new issue #1720.