403 request signature does not match errors with 1.5.0

[fomojola]

I have a Google Cloud Function that uses file.getSignedPolicy() and nodejs 6.11. In the package.json I was pointing to 1.2.0 like this:

"@google-cloud/storage": "^1.2.0",

After recently updating the function my browser form POST uploads using getSignedPolicy started failing with the following error:

<?xml version='1.0' encoding='UTF-8'?><Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.</Message><StringToSign>eyJleHBpcmF0aW9uIjoiMjAxNy0xMS0yOVQyMjoxNjoyMi4wMDhaIiwiY29uZGl0aW9ucyI6W1siZXEiLCIka2V5IiwiYjc2MzBhZjAtOThjNS00YjgwLTg2OWItMDAwMGY1MWZhNDJlLXJlZGJlYXItc3RvcmUucGRmIl0seyJidWNrZXQiOiJmYXhyb2NrZXQtZGF0YSJ9LFsiZXEiLCIkQ29udGVudC1UeXBlIiwiYXBwbGljYXRpb24vcGRmIl0seyJzdWNjZXNzX2FjdGlvbl9zdGF0dXMiOiIyMDAifSxbImNvbnRlbnQtbGVuZ3RoLXJhbmdlIiwwLDIwMDAwMDAwXV19</StringToSign></Error>

Absolutely no change to the cloud function code that was doing the signing or the client upload code. After about an hour of mucking around, I updated the package.json to pin the version to 1.2.0:

"@google-cloud/storage": "1.2.0",

Everything works now. I don't have a clean way to be able to tell what the exact signature difference is, but I looked and realized there was a change in 1.5.0 to the getSignedPolicy function: it is doing something different when deployed in the Google Cloud Functions environment.

As an extra twist, the exact same code with 1.5.0 works when deployed in the function emulator: unfortunately I couldn't figure out how to compare the policy signature output of both versions side by side for the same input.

[stephenplusplus]

Sorry for the trouble, and thank you for sharing what you found. I will try to reproduce.

[stephenplusplus]

Would you mind doing a fresh deploy to GCF using @google-cloud/storage@^1.5.0? I made a change to the way errors are reported when signing fails.

[fomojola]

The issue isn't actually with the signing failing, its that from 1.4.0 to 1.5.0 the actual policy output changed. I finally buckled down and created a simple test:

var storage = require('@google-cloud/storage');
var path = require('path');
var gcs = storage({
    projectId: "test",
    keyFilename: path.join(__dirname, "key.json")
});

var options = {
    'key': "1.key",
    equals: [['$Content-Type', 'application/pdf'], ['$success_action_status','200']],
    successStatus: '200',
    expires: '2018-12-01T06:31:42.096Z',
    contentLengthRange: {
        min: 0,
        max: 20000000
    },
};
var file = gcs.bucket("test").file("1.key");
file.getSignedPolicy(options).then(function(val){
    console.log(val);
});

If you copy this code into a directory and call it test.js, and then copy a private key JSON file (downloaded from the google cloud console) into the same directory with the filename key.json, you can do the following to see the change:

[~/projects/bug] npm install @google-cloud/storage@1.4.0
+ @google-cloud/storage@1.4.0
added 1 package and updated 1 package in 3.088s
[~/projects/bug] node test.js 
[ { string: '{"expiration":"2018-12-01T06:31:42.096Z","conditions":[["eq","$key","1.key"],{"bucket":"test"},["eq","$Content-Type","application/pdf"],["eq","$success_action_status","200"],{"success_action_status":"200"},["content-length-range",0,20000000]]}',
    base64: 'eyJleHBpcmF0aW9uIjoiMjAxOC0xMi0wMVQwNjozMTo0Mi4wOTZaIiwiY29uZGl0aW9ucyI6W1siZXEiLCIka2V5IiwiMS5rZXkiXSx7ImJ1Y2tldCI6InRlc3QifSxbImVxIiwiJENvbnRlbnQtVHlwZSIsImFwcGxpY2F0aW9uL3BkZiJdLFsiZXEiLCIkc3VjY2Vzc19hY3Rpb25fc3RhdHVzIiwiMjAwIl0seyJzdWNjZXNzX2FjdGlvbl9zdGF0dXMiOiIyMDAifSxbImNvbnRlbnQtbGVuZ3RoLXJhbmdlIiwwLDIwMDAwMDAwXV19',
    signature: 'fdjZmFVMjqptIC9k4RFQdsijia8agVZYELpUtr7WCxxaxXzJvSrp/w+ECU7Kke9+Y/W5TxbB/ybhjZPrmA7g0m1WVVh5z+KYixz1CPs7P1ESKVuoH86FSXntYkqIoKgH8b3WKpFGkez9nI1ZYOV7oIvHpYWtFvvkkr61Pk1P/PTD5wXIUF+edWRtYbp0oA7xkhxzldMDk2xy6EkCF8obdqL/utrBi8V9yD2M56uOAhZah/R03ybJqMFxwWlZhM/0Usiq7qbp6lH8AfHYRktYcAtvl84ssgUuEzxSL0dOWj6LNalmFjSpN6TBd8It+Ma7ohqSQZZ2C0TQ9J7Yo7fzqg==' } ]
[~/projects/bug] npm install @google-cloud/storage@1.5.0
+ @google-cloud/storage@1.5.0
added 5 packages and updated 1 package in 4.706s
[~/projects/bug] node test.js 
[ { string: '{"expiration":"2018-12-01T06:31:42.096Z","conditions":[["eq","$key","1.key"],{"bucket":"test"},["eq","$Content-Type","application/pdf"],["eq","$success_action_status","200"],{"success_action_status":"200"},["content-length-range",0,20000000]]}',
    base64: 'eyJleHBpcmF0aW9uIjoiMjAxOC0xMi0wMVQwNjozMTo0Mi4wOTZaIiwiY29uZGl0aW9ucyI6W1siZXEiLCIka2V5IiwiMS5rZXkiXSx7ImJ1Y2tldCI6InRlc3QifSxbImVxIiwiJENvbnRlbnQtVHlwZSIsImFwcGxpY2F0aW9uL3BkZiJdLFsiZXEiLCIkc3VjY2Vzc19hY3Rpb25fc3RhdHVzIiwiMjAwIl0seyJzdWNjZXNzX2FjdGlvbl9zdGF0dXMiOiIyMDAifSxbImNvbnRlbnQtbGVuZ3RoLXJhbmdlIiwwLDIwMDAwMDAwXV19',
    signature: 'hz/C3zdCAQfAJYCsPJ/7mC9R3PwBIDXHv7N6+eTDPNBKrdri8d89aU/Pt6DDaMhkiXgjnGx4BNLKGj6//Zrkv1vLg0PYKIky4r9u7XS6Pg+NnY2TzShmZVzuJ4sCyzcEZeJTKKIWrn1BgQJvrzeDs0ZXghJBEpknO4dMjrzkl7Op6LS/Prl1Cmv9xZ7GDY4EHleq210UR16E1s4PWWhGwAwuJT6FTJM21zQPrgb+BhAJvq8iDOcPS0Cc4HFER7slkyCB847FyezDEEP4/x2H9XoHguFZuuUDBs3IjwAvpQbIOqU2qEUvGYB4KL8XuKk+tB78pQuulrWl3F/CySyGlw==' } ]

You'll notice that for the exact same policy document and base64 data, the signature is different going from 1.4.0 to 1.5.0: there is no error, just a different signature value that isn't accepted by the cloud storage servers.

[stephenplusplus]

Thanks for that. Indeed, we changed how we were generating the signed policy. I have sent a fix in #99.