Unable to read files

[marad]

Trying to read file that I've just written to s3mock gives me empty InputStream.

client.getObject("bucket", "file").getObjectContent() // <- this is empty InputStream

[shuttie]

@marad Can you provide more concrete & full code example for this failure? As I cannot reproduce this issue.

[marad]

Sure, I'll create small example code and send it here.

[marad]

@shuttie Ok, I've found the cause of my problems. In my code I need to create connection with:

val clientConfig = new ClientConfiguration()
clientConfig.setSignerOverride("S3SignerType")
val client = new AmazonS3Client(credentials, clientConfig)

And this signer override is causing the files to be empty. Is there a way to support other signing algorithms in s3mock?

[marad]

@shuttie The problem is, really that the files are stored empty in the temporary directory when putting objects. Setting signer type is one way to achieve this. I'm suspecting other ways but will investigate later.

[sl33nyc]

@marad, I had a similar problem using S3Mock for Presigned URLs. I originally used AnonymousAWSCredentials in my tests but the code path for presigned URLs requires an access key. I found your earlier comment and switched to BasicAWSCredentials and overrode the signer per your comment and repro'd the same "zero bytes" problem.

It turns out that "S3SignerType" is incorrect. Both "AWS3SignerType" and "NoOpSignerType" work for me. I also can verify that the files are no longer zero bytes in size.

[C-Otto]

We experience the same issue, caused by using S3SignerType. I suggest to change the name of this issue accordingly, and I'd appreciate a fix.

[C-Otto]

The code expects the PUT payload after a chunk signature (example below). According to https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html this seems to be specific to V4. With V2 the client only sends the raw data, causing s3mock to fail because of the missing chunk signature.

V4:

org.apache.http.wire - http-outgoing-0 >> "3;chunk-signature=0d1d8b31c94d5e4effc1acb4a97632ae10355cf018fb454eb5400faa5be61bec[\r][\n]"
org.apache.http.wire - http-outgoing-0 >> "FOO[\r][\n]"
org.apache.http.wire - http-outgoing-0 >> "0;chunk-signature=4b2cc53ba80357f42d34d77d596cc47552d586bfa71fb506c0ecf94d9854723e[\r][\n]"

V2 is just org.apache.http.wire - http-outgoing-0 >> "FOO".

The failing logic is in S3ChunkedProtocolStage. In PutObject the S3ChunkedProtocolStage seems to be hardcoded in completeSigned.