Content-Length header is not set when using the form library #316

Closed
mikehaertl opened this Issue Sep 3, 2012 · 18 comments

Projects

None yet

I tried to upload a file using the form library like this:

var r = request.post(options, cb),
    form = r.form();

form.append('image', fs.createReadStream('/tmp/testimg.jpg'));

My server (nginx as reverse proxy in front of Apache) then throws a 411. I then tried to set the Content-Length manually like this:

var r = request.post(options, cb),
    form = r.form();

form.append('image', fs.createReadStream('/tmp/testimg.jpg'));
r.setHeader('Content-Length', form.getLengthSync());

But then the server script (PHP) throws a UPLOAD_ERR_PARTIAL (The file was only partially uploaded) error. So i guess, the Content-Length is still wrong.

fasmide commented Sep 10, 2012

I found a work around for this, having the same problem.

var FormData = require('form-data');
var form = new FormData();
form.append('api_key', api_key);
form.append('product_id', 19);
form.append('file', fs.createReadStream(path.join(__dirname, '/IMG_0859.png')));
form.submit(uri);

uri just being a string url e.g. http://whatever.com/takeImage

Not sure if this is a bug with form-data or request

Yeah, i also considered that. I only hesitated to install form-data as it says "Sorry, this isn't ready for you yet." in their installation comment. But on the other hand, if request is using it anyway, it shouldn't hurt.

Another problem with this workaround is, that i could not get cookies to work.

Owner
mikeal commented Sep 17, 2012

how does it compute the length when streams are added?

Hmm, i thought, you included form-data into your package? So if you can't tell - who can? :)

I'm not an expert on node.js, but maybe a look at their source code helps?
https://github.com/felixge/node-form-data/blob/master/lib/form_data.js

mkopala commented Nov 10, 2012

I'm having the same issue. I was looking at using the workaround @fasmide mentioned, but you can't directly set the headers with the FormData.submit() method either - and I also need to send cookies.

The only way the Content-Length header gets set with request in main.js is if you set body in the options (AFAICT). It won't take a Stream though :-(

I was really hoping I wouldn't have to hack either request for form-data, but it doesn't look like there's a clean way to do this.

@mikehaertl Your first idea of setting the Content-Length manually worked for me.

Owner
mikeal commented Nov 16, 2012

should loop @felixge in to this conversation.

would be nice if form-data set the content-length header when none of the values are streams.

felixge commented Nov 16, 2012

@mikeal @mikehaertl I added you as collaborators on the form-data repo. I'm not going to have time to make a patch right away, so I figured I may was well not make myself the bottleneck here.

@mikeal I also added you as an npm author, so you can publish new versions as you see fit.

@mikehaertl I tried adding you on npm, but it seems like you're using a different username there. Please let me know which one it is, and I'll add you as well.

This WILL work with cookie and correct Content-Length !

you need to: npm install form-data

var FormData = require('form-data');
var request = require('request')
var form = new FormData();
form.append('api_key', api_key);
form.append('product_id', 19);
form.append('file', fs.createReadStream(path.join(__dirname, '/IMG_0859.png')));

form.getLength(function(err,length){
    var r = request.post('http://your-site.com', { headers: { 'content-length': length } }, function(err, res, body){ console.log(body) });
     r._form = form
});

I got a mix of @mikehaertl and @lildemon solutions to work properly:

var r = request.post(options, cb),
    form = r.form();

form.append('image', fs.createReadStream('/tmp/testimg.jpg'));

form.getLength(function(err, length) {
  r.setHeader('Content-Length', length);
});

For some reason, in my tests form.getLength (source) returns the right length whereas form.getLengthSync(source) returns length minus 12

tutukin commented May 25, 2013

The method by marcelduran does not work for me. It looks like r.setHeader is called too late and the header is not actually sent.

So, for every stream one has to specify the length:

    // ....
    form.append('image', fs.createReadStream('/tmp/testimg.jpg'), {knownLength:234567});
    r.setHeader('Content-Length', form.getLengthSync());
    // ...

I believe that the solution by @lildemon is the only one which avoids a race condition.

There's no way to guarantee that the form.getLength's callback executes before request.post, unless one executes request.post within the getLength callback.

@aslinwang aslinwang referenced this issue in aslinwang/increjs Jul 28, 2014
Open

`incre upload`命令使用说明 #1

Owner
mikeal commented Aug 27, 2014

Is this still an issue?

This is so old I'm closing, if it is actually still an issue just let me know and I'll re-open.

@mikeal mikeal closed this Aug 27, 2014

Yo, 2015 and I've been struggling with this for a few hours now. I have a npm module that takes a stream, a filename, and a stream length, and writes the stream contents to cloud storage.

I have a server that accepts form uploads, and uses multiparty on the form to pass the file stream, filename, and stream length to the module.

I also have a programmatic client that creates a form (using request.js and form-data) that it uses to upload files to the server, which ultimately end up in cloud storage.

It was really frustrating that a simple html web page with one file input can upload a file to the server which multiparty handles flawlessly, but any programmatic usage of request to create a form post my server has been a PITA.

Even if I set the "knownLength" option on the part that contains my stream, the content-length header for the part is never set, so whenever I try to get the byteCount from multiparty it isn't there because in multiparty land the byteCount comes from the content-length header. FIRST WORLD PROBLEMS :P

Anywho, I made a pull-request, form-data/form-data#120, for form-data that I tested with my setup and I'm happy with it. My issue is the content-length not being set on the individual parts, not sure if that's the same issue y'all are having (pretty sure it isn't, but if the PR goes in I'd want to access it via request.js, not form-data directly).

I also want to say that I'm having the same problem, and I think this should be open. I want to implement a progress indicator for an upload. I would love any input about how to do that-- not having content length seems to make it quite tough.

qnimate commented Oct 1, 2015

I am using multiparty and request module together. Here is code example:

app.post('/submit', function(httpRequest, httpResponse, next){

    var form = new multiparty.Form();

    form.on("part", function(part){
        if(part.filename)
        {

            var FormData = require('form-data');
            var form = new FormData();
            form.append('thumbnail', part);

            form.getLength(function(err,length){
                console.log(length);
                var r = request.post('http://localhost:8888/index.php', { headers: { 'content-length': length } }, function(err, res, body){ 
                    if(err) {
                        return console.error('upload failed:', err);
                    }

                    console.log(body) 
                });
                 r._form = form
            });
        }
    })

    form.on("error", function(error){
        console.log(error);
    })

    form.parse(httpRequest);    
});

Here I am uploading the user uploaded file to another server via streaming i.e., without saving a temporary file.

But on the PHP server no file is uploaded. $_FILES array is empty.

Please help.

@nmccready nmccready referenced this issue in CartoDB/cartodb-nodejs Sep 26, 2016
Open

Import API Streaming [feature Request] #57

Petah commented Oct 27, 2016

+1 still not working...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment