need v2 signing to create compatible signed url for DO

[sam0x17]

(using node.js in a Google Cloud Function to create a putObject presigned URL in a DigitalOcean Spaces bucket)

It looks like support has been dropped from aws-sdk for doing presigned URLs that use the v2 signing version. When I specify signatureVersion: "v2" I get UnsupportedSigner: Presigning only supports S3 or SigV4 signing.. Is there a way I can generate a presigned url using the v2 signature version? I don't mind if I need to use an old version of aws-sdk or even another npm package -- just looking for a way to make this work.

[chrisradek]

@sam0x17
What version of the SDK are you using? If you don't specify the signatureVersion of your S3 client, then the S3 client will create a presigned URL using 's3' by default. (Specifying 'v2' also uses 's3'). The only exception to this rule is if the region you've set does not support signatureVersion v2.

I just tested generating 'v2' signed URLs using the latest version of the SDK, and so far haven't been able to reproduce your issue.

// force signature version v2
var url = new AWS.S3({signatureVersion: 'v2'}).getSignedUrl('putObject', {Bucket: 'BUCKET', Key: 'KEY'})

[sam0x17]

@chrisradek ok I tried it that way and now I actually get a signed URL back (before I was putting signatureVersion in the wrong place, turns out), however I am still getting SignatureDoesNotMatch when I try to PUT to the signed URL

Here is what I am putting in to my rest client:

Maybe there is some header I am supposed to be sending that I am not sending??

I am using the latest version of the aws-sdk npm package, i.e. I ran npm install aws-sdk --save this morning.

[sam0x17]

@chrisradek here is my code:

const secureRandom = require('secure-random');
var AWS = require('aws-sdk');
AWS.config.loadFromPath('./credentials.json');
var s3 = new AWS.S3({signatureVersion: 'v2'});

function randomAlphaNumeric(size) {
	return secureRandom(size * 2, {type: 'Buffer'})
	  .toString('base64')
		.replace(/[\/+=]/g, '')
		.substring(0, size);
}

exports.requestUploadUrl = function requestUploadUrl(req, res) {
  var key = randomAlphaNumeric(16);
  s3.getSignedUrl('putObject', {Key: key, Bucket: 'bitfort', Expires: 1000000}, function (err, url) {
    res.send(url);
    console.log('The URL is', url);
    if(err) throw err;
  });
}

[chrisradek]

@sam0x17
Glad to hear you're able to generate the signed URLs.

Can you test your URLs against S3 itself? Unfortunately, since your tests are against a 3rd party service there's not much I can do to determine why you're receiving that error. The issue may be with the service itself.

If you have the same issue when making requests against Amazon S3, then I'd be happy to help!

[sam0x17]

@chrisradek I just tried with my s3 bucket and got the same issue, so I don't think it's specific to digitalocean.

response from s3:

<Error>
<Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
<AWSAccessKeyId>*****************</AWSAccessKeyId>
<StringToSign>PUT multipart/form-data; boundary=----WebKitFormBoundaryzaMQH9PWv0AwF7K5 1509876896 /blockvue.datastore/some-key</StringToSign>
<SignatureProvided>IEyfPAkg3OXO+Wi9w63CYrJSYsc=</SignatureProvided>
<StringToSignBytes>50 55 54 0a 0a 6d 75 6c 74 69 70 61 72 74 2f 66 6f 72 6d 2d 64 61 74 61 3b 20 62 6f 75 6e 64 61 72 79 3d 2d 2d 2d 2d 57 65 62 4b 69 74 46 6f 72 6d 42 6f 75 6e 64 61 72 79 7a 61 4d 51 48 39 50 57 76 30 41 77 46 37 4b 35 0a 31 35 30 39 38 37 36 38 39 36 0a 2f 62 6c 6f 63 6b 76 75 65 2e 64 61 74 61 73 74 6f 72 65 2f 73 6f 6d 65 2d 6b 65 79</StringToSignBytes>
<RequestId>5B058BD7B72D1719</RequestId>
<HostId>2UbgHN+ECXQ5nb7EVX9Ci1vhNNFOGdF7Dh5ppCBUkfrKQQuJ6BCbbI3uuCvscGibOuamyCKV8d4=</HostId>
 </Error>

[chrisradek]

@sam0x17
I notice now that you're trying to upload FormData. That will cause corrupted data to be uploaded.
See this and this.

You would either need to upload the binary (you should be able to specify a Blob as the body), or use a presigned Post instead.

[sam0x17]

ah, that is probably my problem -- thank you!

[sam0x17]

@chrisradek ok I changed it to upload just a blob. I tried "any file data" (no content type) and application/octet-stream, still got the same error for both:

[sam0x17]

@chrisradek I was able to get it to work by specifying ContentType: 'application/octet-stream' both in the signed url and in the PUT request. That said, I need to upload enormous files so I need chunked/multi-part. Is that going to work with a PUT request, or should I be using POST?

[chrisradek]

@sam0x17
Ah yes, for that you'd want to use a POST.

Glad you were able to get the PUT working though!

[sam0x17]

@chrisradek OK in that case what do I need to name the file field for the POST? will it take anything?

[chrisradek]

@sam0x17
Please see the documentation for s3.createPresignedPost

The callback will return a data object where fields is the hash of fields your form has to include.

There's an example of what an HTML page would look like using the presigned post.

However, it just occurred to me that our implementation of presigned POST only works with signatureVersion v4, so if Digital Oceans Spaces doesn't support v4, then you'd have to resort to using a presigned PUT URL.

[sam0x17]

@chrisradek in that case what is the max size I can squeeze into a single PUT? For my application, it is OK if the data is spread out across multiple objects on the server

[chrisradek]

That's going to depend on what the service accepts.

For S3, their FAQ states:

The largest object that can be uploaded in a single PUT is 5 gigabytes.

[sam0x17]

thanks

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.