PutObject creates empty file when using the JS AWS SDK

[scooper91]

This creates an empty file at /root/my-bucket/some-key/fileData:

await s3.putObject({
  Bucket: 'my-bucket',
  Key: 'some-key',
  Body: 'some contents'
}).promise()

We've tried different body types, as well as specifying ContentType and ContentLength, with the same results.

It seems related to ContentLength as when uploading a file using curl, the file is empty when a Content-Length header is not provided.

[psxpaul]

This appears to be a bug when using the aws cli as well:

[proberts]:[~/Downloads]$ aws s3api put-object --endpoint-url http://localhost:9090 --bucket test-bucket --key testone --body test.txt
{
    "ETag": "\"d41d8cd98f00b204e9800998ecf8427e\""
}
[proberts]:[~/Downloads]$ aws s3api list-objects-v2 --endpoint-url http://localhost:9090 --bucket test-bucket
{
    "Contents": [
        {
            "Key": "testone",
            "LastModified": "2019-01-31T18:33:47.000Z",
            "ETag": "d41d8cd98f00b204e9800998ecf8427e",
            "Size": 0,
            "StorageClass": "STANDARD",
            "Owner": {
                "DisplayName": "s3-mock-file-store",
                "ID": "123"
            }
        }
    ],
    "CommonPrefixes": [
        {}
    ]
}

I tried setting the content-type, content-length, and content-encoding to no avail. I also tried the docker image versions 2.1.0, 2.0.0, 1.1.0, and 1.0.0. Earlier versions throw an error about not being able to write a file, but newer versions just save a 0 byte file.

[junglie85]

I have the same issue when uploading using boto3 (Python AWS client).

[matjamesymj]

I seem to be getting the same issue when using the ruby gem sdk using the below code:

s3 = Utilities.awsConnect
  resources = Utilities.awsResources(s3)
  path =  '/fakepath/fakefile.csv'
  folder_location = resources.bucket('fakebucket').object(path)
  folder_location.upload_file('fakepath/fakefile.csv')

If I upload using java sdk using the following code it works:

File file = new File("/fakepath/fakefile.csv");
       s3Client.putObject("fakebucket", "fakepath/fakefile.csv", file)

Any ideas?

[haarcuba]

I have the same issue using python's boto3:

In [56]: c.put_object(Key='mykey', Bucket='mybucket', Body=json.dumps({'a':1}) )
Out[56]: 
{'ResponseMetadata': {'HTTPStatusCode': 200,
  'HTTPHeaders': {'date': 'Tue, 30 Apr 2019 21:58:56 GMT',
   'last-modified': 'Tue, 30 Apr 2019 21:58:56 GMT',
   'etag': '"d41d8cd98f00b204e9800998ecf8427e"',
   'content-length': '0',
   'server': 'Jetty(9.4.14.v20181114)'},
  'RetryAttempts': 0},
 'ETag': '"d41d8cd98f00b204e9800998ecf8427e"'}

In [57]: c.get_object(Key='mykey', Bucket='mybucket')['Body'].read()
Out[57]: b''

[BLevinger]

I was able to get to work by using signatureVersion v3
e.g.
AWS.S3({apiVersion: '2006-03-01', signatureVersion: "v3"})

[davidsbond]

Having the same issue using the golang client

[nealstewart]

@davidsbond I was running into the same issue with boto3.

As @haarcuba mentioned, this can be resolved by setting the signature version like so:

boto3client(
    "s3",
    endpoint_url="http://yours3mock",
    config=boto3.session.Config(signature_version="v3"),
)

This only needs to be done for the resource/client that is doing the uploading. Once uploaded properly, the files seem accessible through other operations (list_objects, get... etc)

[davidsbond]

Thanks @nealstewart, I'm unable to find a way of setting the signature version using the aws golang client. I can't seem to find an option for it in the configuration, although it could be named something else. For now I was able to directly mock the S3 bucket locally using another library.

[agudian]

I needed some help with getting a reproducer to work and debug, and after that it turned out that boto3 and the golang client (apparently) used a combination of v4 signing and non-chunked transfer that resulted in the problem.

The S3Mock didn't expect v4 signing without chunking 🤷‍♂.

The fix is on its way...

[SuperStar518]

I was able to get to work by using signatureVersion v3
e.g.
AWS.S3({apiVersion: '2006-03-01', signatureVersion: "v3"})

@BLevinger saved my day! Thanks.