New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

error 403 #1

Closed
amenadiel opened this Issue Jul 21, 2014 · 8 comments

Comments

Projects
None yet
4 participants
@amenadiel

amenadiel commented Jul 21, 2014

I keep geting 403 errors. I thought it was related to special characters in the file name, but it happens even with the simplest filenames.

Is it possible that the problem regards the S3 Region? I'm creating all my buckets in US Standard. ¿Perhaps I should use a specific region?

@amenadiel

This comment has been minimized.

Show comment
Hide comment
@amenadiel

amenadiel Jul 21, 2014

Update:

I finally found that the problem lies in the parameter

req.query.s3_object_type

In my case, the frontend was sending the file type as application/vnd.google-earth.kml+xml

For some reason, express parses it as application/vnd.google-earth.kml xml

From then on, the provided signature was valid for a file type with a space, whereas the front was uploading a file type with a '+' sign.

amenadiel commented Jul 21, 2014

Update:

I finally found that the problem lies in the parameter

req.query.s3_object_type

In my case, the frontend was sending the file type as application/vnd.google-earth.kml+xml

For some reason, express parses it as application/vnd.google-earth.kml xml

From then on, the provided signature was valid for a file type with a space, whereas the front was uploading a file type with a '+' sign.

@willwebberley

This comment has been minimized.

Show comment
Hide comment
@willwebberley

willwebberley Jul 22, 2014

Owner

Thanks for the issue report and for your update. I will have a look through the code and make changes! Are you saying that the mime type works with a space or a '+' sign?

Owner

willwebberley commented Jul 22, 2014

Thanks for the issue report and for your update. I will have a look through the code and make changes! Are you saying that the mime type works with a space or a '+' sign?

@amenadiel

This comment has been minimized.

Show comment
Hide comment
@amenadiel

amenadiel Jul 22, 2014

In the end, I sent the mime type base64-encoded and then decoded in node using npm's btoa module.

I could have forced the '+' too, but I guess there might be another special chars in the mime type that get stripped or replaced by node express, so I played safe.

amenadiel commented Jul 22, 2014

In the end, I sent the mime type base64-encoded and then decoded in node using npm's btoa module.

I could have forced the '+' too, but I guess there might be another special chars in the mime type that get stripped or replaced by node express, so I played safe.

@willwebberley

This comment has been minimized.

Show comment
Hide comment
@willwebberley

willwebberley Jul 23, 2014

Owner

Maybe this is an issue with the s3upload.js script then. Have you tried using the upstream version here? The version in this repo is outdated at the moment!

Owner

willwebberley commented Jul 23, 2014

Maybe this is an issue with the s3upload.js script then. Have you tried using the upstream version here? The version in this repo is outdated at the moment!

@scott-laursen

This comment has been minimized.

Show comment
Hide comment
@scott-laursen

scott-laursen Jul 23, 2014

@amenadiel I am having the same issue, would you maybe be willing to share the code that made it work for you?

I will add that my issue seems to be when the mimetype is blank.

scott-laursen commented Jul 23, 2014

@amenadiel I am having the same issue, would you maybe be willing to share the code that made it work for you?

I will add that my issue seems to be when the mimetype is blank.

@amenadiel

This comment has been minimized.

Show comment
Hide comment
@amenadiel

amenadiel Jul 23, 2014

@scott-laursen this is my not-so-elegant solution
in s3upload.jsmethod executeOnSignedUrl

S3Upload.prototype.executeOnSignedUrl = function(file, callback) {
      var this_s3upload, xhr;
      this_s3upload = this;
      xhr = new XMLHttpRequest();
      xhr.open('GET', this.s3_sign_put_url + '?s3_object_type=' + window.btoa(file.type) + '&s3_object_name=' + this.s3_object_name, true);
      xhr.overrideMimeType('text/plain; charset=x-user-defined');
      xhr.onreadystatechange = function(e) {
        var result;
        if (this.readyState === 4 && this.status === 200) {
          try {
            result = JSON.parse(this.responseText);
          } catch (error) {
            this_s3upload.onError('Signing server returned some ugly/empty JSON: "' + this.responseText + '"');
            return false;
          }
          return callback(decodeURIComponent(result.signed_request), result.url);
        } else if (this.readyState === 4 && this.status !== 200) {
          return this_s3upload.onError('Could not contact request signing server. Status = ' + this.status);
        }
      };
      return xhr.send();
    };

in node's app.js route sign_s3

/*
 * Respond to GET requests to /sign_s3.
 * Upon request, return JSON containing the temporarily-signed S3 request and the
 * anticipated URL of the image.
 */
app.get('/sign_s3', function(req, res){
    var atob = require('atob');
    var object_name = req.query.s3_object_name;
    var mime_type = atob(req.query.s3_object_type);

    var now = new Date();
    var expires = Math.ceil((now.getTime() + 10000)/1000); // 10 seconds from now
    var amz_headers = "x-amz-acl:public-read";  

    var put_request = "PUT\n\n"+mime_type+"\n"+expires+"\n"+amz_headers+"\n/"+S3_BUCKET+"/"+object_name;

    var signature = crypto.createHmac('sha1', AWS_SECRET_KEY).update(put_request).digest('base64');
    signature = encodeURIComponent(signature.trim());
    signature = signature.replace('%2B','+');

    var url = 'https://'+S3_BUCKET+'.s3.amazonaws.com/'+object_name;

    var credentials = {
        signed_request: url+"?AWSAccessKeyId="+AWS_ACCESS_KEY+"&Expires="+expires+"&Signature="+signature,
        url: url
    };
    res.write(JSON.stringify(credentials));
    res.end();
});

note I'm using npm's atob. It's not in the core of node.js

@flyingsparx, except from my window.btoa I haven't changed much. It seems I was already using the version you linked.

amenadiel commented Jul 23, 2014

@scott-laursen this is my not-so-elegant solution
in s3upload.jsmethod executeOnSignedUrl

S3Upload.prototype.executeOnSignedUrl = function(file, callback) {
      var this_s3upload, xhr;
      this_s3upload = this;
      xhr = new XMLHttpRequest();
      xhr.open('GET', this.s3_sign_put_url + '?s3_object_type=' + window.btoa(file.type) + '&s3_object_name=' + this.s3_object_name, true);
      xhr.overrideMimeType('text/plain; charset=x-user-defined');
      xhr.onreadystatechange = function(e) {
        var result;
        if (this.readyState === 4 && this.status === 200) {
          try {
            result = JSON.parse(this.responseText);
          } catch (error) {
            this_s3upload.onError('Signing server returned some ugly/empty JSON: "' + this.responseText + '"');
            return false;
          }
          return callback(decodeURIComponent(result.signed_request), result.url);
        } else if (this.readyState === 4 && this.status !== 200) {
          return this_s3upload.onError('Could not contact request signing server. Status = ' + this.status);
        }
      };
      return xhr.send();
    };

in node's app.js route sign_s3

/*
 * Respond to GET requests to /sign_s3.
 * Upon request, return JSON containing the temporarily-signed S3 request and the
 * anticipated URL of the image.
 */
app.get('/sign_s3', function(req, res){
    var atob = require('atob');
    var object_name = req.query.s3_object_name;
    var mime_type = atob(req.query.s3_object_type);

    var now = new Date();
    var expires = Math.ceil((now.getTime() + 10000)/1000); // 10 seconds from now
    var amz_headers = "x-amz-acl:public-read";  

    var put_request = "PUT\n\n"+mime_type+"\n"+expires+"\n"+amz_headers+"\n/"+S3_BUCKET+"/"+object_name;

    var signature = crypto.createHmac('sha1', AWS_SECRET_KEY).update(put_request).digest('base64');
    signature = encodeURIComponent(signature.trim());
    signature = signature.replace('%2B','+');

    var url = 'https://'+S3_BUCKET+'.s3.amazonaws.com/'+object_name;

    var credentials = {
        signed_request: url+"?AWSAccessKeyId="+AWS_ACCESS_KEY+"&Expires="+expires+"&Signature="+signature,
        url: url
    };
    res.write(JSON.stringify(credentials));
    res.end();
});

note I'm using npm's atob. It's not in the core of node.js

@flyingsparx, except from my window.btoa I haven't changed much. It seems I was already using the version you linked.

@willwebberley

This comment has been minimized.

Show comment
Hide comment
@willwebberley

willwebberley Apr 27, 2015

Owner

Apologies for the late reply - been pretty swamped and I was unable to prioritise this any higher!

I have now removed the need for client-side dependencies. have done some pretty vigorous tests, and have not encountered any 403 errors.

@scott-laursen: The mime-type of the file set when signing the request must be the same as the mime-type set by your browser during the upload, since this value is used to re-calculate the signature on the AWS end. The latest version of the code in this repository should now address that. If the mime type cannot be automatically extracted, you will need to manually set the mine-type sent in the headers of the PUT request and that used to sign the request on the app to the same thing.

Owner

willwebberley commented Apr 27, 2015

Apologies for the late reply - been pretty swamped and I was unable to prioritise this any higher!

I have now removed the need for client-side dependencies. have done some pretty vigorous tests, and have not encountered any 403 errors.

@scott-laursen: The mime-type of the file set when signing the request must be the same as the mime-type set by your browser during the upload, since this value is used to re-calculate the signature on the AWS end. The latest version of the code in this repository should now address that. If the mime type cannot be automatically extracted, you will need to manually set the mine-type sent in the headers of the PUT request and that used to sign the request on the app to the same thing.

@MindfulMe

This comment has been minimized.

Show comment
Hide comment
@MindfulMe

MindfulMe May 13, 2018

thanks for the clarification

MindfulMe commented May 13, 2018

thanks for the clarification

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment