411 Length Required Error #396

Closed
gahula opened this Issue Dec 27, 2012 · 10 comments

Comments

Projects
None yet
3 participants
@gahula

gahula commented Dec 27, 2012

The code uploads file(s) to a server. It worked while I was on localhost; however, it doesn't work now I have transferred to the production server. I get the error 411 Length Required , I assume this is the content length, when I do specify a length it no longer gives an error but the file upload process never starts.

var request = require('request');
var fs = require('fs');
var path = require('path');

   options.method = 'POST';
options.uri = 'http://subdomain.domain.com/username;

var cdn = request(options, function(error, response, body) {
    if(error) {
        console.log(error);
        res.send(500);
    } else {
        if(response) {
            res.headers = response.headers;
        }
        if(body) {
            res.send(response.statusCode, body);
        }
    }
});

var form = cdn.form();

_.each(req.files.files, function (file) {
    var i = file.name.lastIndexOf('.');
    var ext = file.name.substr(i);
    var path = file.path + ext;
        form.append(ext, fs.createReadStream(file.path)); 
}); 

I tried the method shown in your example using the request.post function
...

var r = request.post('http://service.com/upload')
var form = r.form()
form.append('my_field', 'my_value')
form.append('my_buffer', new Buffer([1, 2, 3]))
form.append('my_file', fs.createReadStream(path.join(__dirname, 'doodle.png'))
form.append('remote_file', request('http://google.com/doodle.png'))

... nothing happens... the file upload process never starts.

@mikeal

This comment has been minimized.

Show comment Hide comment
@mikeal

mikeal Dec 27, 2012

Member

yeah, transfer encoding is chunked when using streaming. also, i'm pretty sure that the form library doesn't set a content-length for uploads, there is a ticket for that somewhere in that project's github repo. what service are you uploading to?

Member

mikeal commented Dec 27, 2012

yeah, transfer encoding is chunked when using streaming. also, i'm pretty sure that the form library doesn't set a content-length for uploads, there is a ticket for that somewhere in that project's github repo. what service are you uploading to?

@gahula

This comment has been minimized.

Show comment Hide comment
@gahula

gahula Dec 27, 2012

My own file server, it works perfectly on my localhost though... i don't know... been on this for the past 24Hrs.
Is there a file upload alternative. I've tried the piping... that didn't work out too well.

gahula commented Dec 27, 2012

My own file server, it works perfectly on my localhost though... i don't know... been on this for the past 24Hrs.
Is there a file upload alternative. I've tried the piping... that didn't work out too well.

@mikeal

This comment has been minimized.

Show comment Hide comment
@mikeal

mikeal Dec 27, 2012

Member

if it's your own upload server then you're returning a 411. what library are you using to parse the multipart?

also, if you're just uploading one file, don't use multipart, just do a PUT and make the entire resource be the file you're sending.

-Mikeal

On Dec 27, 2012, at December 27, 201212:12 PM, gahula notifications@github.com wrote:

My own file server, it works perfectly on my localhost though... i don't know... been on this for the past 24Hrs.
Is there a file upload alternative. I've tried the piping... that didn't work out too well.


Reply to this email directly or view it on GitHub.

Member

mikeal commented Dec 27, 2012

if it's your own upload server then you're returning a 411. what library are you using to parse the multipart?

also, if you're just uploading one file, don't use multipart, just do a PUT and make the entire resource be the file you're sending.

-Mikeal

On Dec 27, 2012, at December 27, 201212:12 PM, gahula notifications@github.com wrote:

My own file server, it works perfectly on my localhost though... i don't know... been on this for the past 24Hrs.
Is there a file upload alternative. I've tried the piping... that didn't work out too well.


Reply to this email directly or view it on GitHub.

@mikeal

This comment has been minimized.

Show comment Hide comment
@mikeal

mikeal Dec 27, 2012

Member

also, where are you deploying this?

Member

mikeal commented Dec 27, 2012

also, where are you deploying this?

@gahula

This comment has been minimized.

Show comment Hide comment
@gahula

gahula Dec 27, 2012

I am using https://github.com/blueimp/jQuery-File-Upload and I have changed the code, to suit my requirements, from the rudimentary server that was provided, I have placed it in express and I use it for simple photo uploads.

It is currently in a sandbox space. but i am running node on WebFaction -- runs on a shared host.
No it is not the server. Even when that is not running (my file server) I get the same message. It is the client, the length just isn't present.

The trick i did on my localhost was to relay the headers. So the headers that came in with the ajax request and file(s), i placed into my file server request so... it was as if the browser was talking directly to the file server. That method now gives me { [Error: socket hang up] code: 'ECONNRESET' }.

I tried your file upload method that now gives 411 Length Required Error. I'll keep trying tonight.

<title>411 Length Required</title>

411 Length Required


nginx

gahula commented Dec 27, 2012

I am using https://github.com/blueimp/jQuery-File-Upload and I have changed the code, to suit my requirements, from the rudimentary server that was provided, I have placed it in express and I use it for simple photo uploads.

It is currently in a sandbox space. but i am running node on WebFaction -- runs on a shared host.
No it is not the server. Even when that is not running (my file server) I get the same message. It is the client, the length just isn't present.

The trick i did on my localhost was to relay the headers. So the headers that came in with the ajax request and file(s), i placed into my file server request so... it was as if the browser was talking directly to the file server. That method now gives me { [Error: socket hang up] code: 'ECONNRESET' }.

I tried your file upload method that now gives 411 Length Required Error. I'll keep trying tonight.

<title>411 Length Required</title>

411 Length Required


nginx
@mikeal

This comment has been minimized.

Show comment Hide comment
@mikeal

mikeal Dec 27, 2012

Member

you don't seem to understand what is happening.

multipart uploads do not require, by spec, a content-length. they can use chunked encoding. that's what this client is using.

someone in this transaction requires a content-length for multipart for some reason only known to them. since it runs on localhost just fine I would say that it's probably WebFaction's load balancer or reverse proxy that requires a content-length.

the multipart library used by request does not add a content-length when you add a stream. you'll need to either work around this issue by using another hosting service or changing the upload type or you can take a crack at adding content-length for this use case in the underlying multipart library.

Member

mikeal commented Dec 27, 2012

you don't seem to understand what is happening.

multipart uploads do not require, by spec, a content-length. they can use chunked encoding. that's what this client is using.

someone in this transaction requires a content-length for multipart for some reason only known to them. since it runs on localhost just fine I would say that it's probably WebFaction's load balancer or reverse proxy that requires a content-length.

the multipart library used by request does not add a content-length when you add a stream. you'll need to either work around this issue by using another hosting service or changing the upload type or you can take a crack at adding content-length for this use case in the underlying multipart library.

@mikeal mikeal closed this Dec 27, 2012

@mikeal

This comment has been minimized.

Show comment Hide comment
@mikeal

mikeal Dec 27, 2012

Member

i noticed you're uploading to a CDN, many CDN's require a content-length. can you tell me which CDN and what API endpoint you're using.

Member

mikeal commented Dec 27, 2012

i noticed you're uploading to a CDN, many CDN's require a content-length. can you tell me which CDN and what API endpoint you're using.

@gahula

This comment has been minimized.

Show comment Hide comment
@gahula

gahula Dec 28, 2012

Thanks for replying, persistence has paid off.
I am assuming you think I am using Akamai or Cachefly or Amazon , etc... CDN is just the name of a variable. i am not using an API or CDN. I am using a Node Server to receive files and store them on the server in a directory. I called it CDN since I will later expand it to handle other file types.

I got it to work by the relay method i mentioned

My Solution: I had to use localhost and port number explicitly and not the url of my server. problem solved!
That is all the code I use below for the client. You can break the headers down explicitly to edit, add or subtract as you wish.

//_Headers_****
/*options.headers['host'] = 'www.yourdomain.com';
options.headers['x-forwarded-host'] = 'www.yourdomain.com';
options.headers['x-forwarded-server'] = 'www.yourdomain.com';
options.headers['x-forwarded-for'] = req.headers['x-forwarded-for'];
options.headers['forwarded-request-uri'] = req.headers['forwarded-request-uri'];
options.headers['http-x-forwarded-proto'] = req.headers['http-x-forwarded-proto'];
options.headers['https'] = req.headers['https'];
options.headers['x-forwarded-proto'] = req.headers['x-forwarded-proto'];
options.headers['x-forwarded-ssl'] = req.headers['x-forwarded-ssl'];
options.headers['connection'] = req.headers['connection'];
options.headers['user-agent'] = req.headers['user-agent'];
options.headers['accept'] = req.headers['accept'];
options.headers['accept-language'] = req.headers['accept-language'];
options.headers['accept-encoding'] = req.headers['accept-encoding'];
options.headers['x-requested-with'] = req.headers['x-requested-with'];
options.headers['referer'] = req.headers['referer'];
options.headers['content-length'] = req.headers['content-length'];
options.headers['content-type'] = req.headers['content-type'];
options.headers['cookie'] = req.headers['cookie'];
options.headers['pragma'] = req.headers['pragma'];
options.headers['cache-control'] = req.headers['cache-control'];
*/

THANKS for replying.

var request = require('request');
var fs = require('fs');
var path = require('path');

var options = {
headers: {}
};

   options.headers = req.headers; // RELAYS THE AJAX REQUEST
options.method = 'POST';
options.uri = 'http://localhost:#####/' + username;// username comes from a session variable

var cdn = request(options, function(error, response, body) {
    if(error) {
        console.log(error);
        res.send(500);
    } else {
        if(response) {
            res.headers = response.headers;// RELAYS TO BROWSER FROM FILESERVER
        }
        if(body) {
            res.send(response.statusCode, body);// RELAYS TO BROWSER FROM FS
        }
    }
});

var form = cdn.form();

_.each(req.files.files, function (file) {
    var i = file.name.lastIndexOf('.');
    var ext = file.name.substr(i);
    var path = file.path + ext;
    var error = validate(file, ext);
    if(error) {
        console.log(error);
        res.send(500);
    } else {
        form.append(ext, fs.createReadStream(file.path)); 
    }
});

gahula commented Dec 28, 2012

Thanks for replying, persistence has paid off.
I am assuming you think I am using Akamai or Cachefly or Amazon , etc... CDN is just the name of a variable. i am not using an API or CDN. I am using a Node Server to receive files and store them on the server in a directory. I called it CDN since I will later expand it to handle other file types.

I got it to work by the relay method i mentioned

My Solution: I had to use localhost and port number explicitly and not the url of my server. problem solved!
That is all the code I use below for the client. You can break the headers down explicitly to edit, add or subtract as you wish.

//_Headers_****
/*options.headers['host'] = 'www.yourdomain.com';
options.headers['x-forwarded-host'] = 'www.yourdomain.com';
options.headers['x-forwarded-server'] = 'www.yourdomain.com';
options.headers['x-forwarded-for'] = req.headers['x-forwarded-for'];
options.headers['forwarded-request-uri'] = req.headers['forwarded-request-uri'];
options.headers['http-x-forwarded-proto'] = req.headers['http-x-forwarded-proto'];
options.headers['https'] = req.headers['https'];
options.headers['x-forwarded-proto'] = req.headers['x-forwarded-proto'];
options.headers['x-forwarded-ssl'] = req.headers['x-forwarded-ssl'];
options.headers['connection'] = req.headers['connection'];
options.headers['user-agent'] = req.headers['user-agent'];
options.headers['accept'] = req.headers['accept'];
options.headers['accept-language'] = req.headers['accept-language'];
options.headers['accept-encoding'] = req.headers['accept-encoding'];
options.headers['x-requested-with'] = req.headers['x-requested-with'];
options.headers['referer'] = req.headers['referer'];
options.headers['content-length'] = req.headers['content-length'];
options.headers['content-type'] = req.headers['content-type'];
options.headers['cookie'] = req.headers['cookie'];
options.headers['pragma'] = req.headers['pragma'];
options.headers['cache-control'] = req.headers['cache-control'];
*/

THANKS for replying.

var request = require('request');
var fs = require('fs');
var path = require('path');

var options = {
headers: {}
};

   options.headers = req.headers; // RELAYS THE AJAX REQUEST
options.method = 'POST';
options.uri = 'http://localhost:#####/' + username;// username comes from a session variable

var cdn = request(options, function(error, response, body) {
    if(error) {
        console.log(error);
        res.send(500);
    } else {
        if(response) {
            res.headers = response.headers;// RELAYS TO BROWSER FROM FILESERVER
        }
        if(body) {
            res.send(response.statusCode, body);// RELAYS TO BROWSER FROM FS
        }
    }
});

var form = cdn.form();

_.each(req.files.files, function (file) {
    var i = file.name.lastIndexOf('.');
    var ext = file.name.substr(i);
    var path = file.path + ext;
    var error = validate(file, ext);
    if(error) {
        console.log(error);
        res.send(500);
    } else {
        form.append(ext, fs.createReadStream(file.path)); 
    }
});
@yyfrankyy

This comment has been minimized.

Show comment Hide comment
@yyfrankyy

yyfrankyy Jun 21, 2013

Contributor

hi @gahula

you should try this:

var FormData = require('form-data')
var f = new FormData()
f.append('uploadfile', fs.createReadStream('/path/to/your/file'))
f.getLength(function(err, length) {
  request.post({
      url: 'http://path/to/your/own/upload/server'
    , headers: {
        'content-length': length
      }
  }, console.log)._form = f
})

Or if you don't need any feature request provided (for example: cookie sharing), just use

var FormData = require('form-data')
var f = new FormData()
f.append('uploadfile', fs.createReadStream('/path/to/your/file'))
f.submit('http://path/to/your/own/upload/server', console.log)
Contributor

yyfrankyy commented Jun 21, 2013

hi @gahula

you should try this:

var FormData = require('form-data')
var f = new FormData()
f.append('uploadfile', fs.createReadStream('/path/to/your/file'))
f.getLength(function(err, length) {
  request.post({
      url: 'http://path/to/your/own/upload/server'
    , headers: {
        'content-length': length
      }
  }, console.log)._form = f
})

Or if you don't need any feature request provided (for example: cookie sharing), just use

var FormData = require('form-data')
var f = new FormData()
f.append('uploadfile', fs.createReadStream('/path/to/your/file'))
f.submit('http://path/to/your/own/upload/server', console.log)
@gahula

This comment has been minimized.

Show comment Hide comment
@gahula

gahula Feb 23, 2014

I actually let the file upload happen from the client's side to the "CDN" now rather than having to use an intermediary (var xhr = new XMLHttpRequest()).

gahula commented Feb 23, 2014

I actually let the file upload happen from the client's side to the "CDN" now rather than having to use an intermediary (var xhr = new XMLHttpRequest()).

LoicMahieu added a commit to LoicMahieu/request that referenced this issue Jul 2, 2014

LoicMahieu added a commit to LoicMahieu/request that referenced this issue Jul 2, 2014

LoicMahieu added a commit to LoicMahieu/request that referenced this issue Jul 8, 2014

nylen pushed a commit to nylen/request that referenced this issue Oct 17, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment