Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Module does not resume/recover from network loss mid get request #2285

Closed
Sigurthorb opened this issue Jul 15, 2016 · 1 comment
Closed

Module does not resume/recover from network loss mid get request #2285

Sigurthorb opened this issue Jul 15, 2016 · 1 comment
Labels

Comments

@Sigurthorb
Copy link

We are using the request@2.52.0 module to download files from a node server.
This has also been tested on request@2.73.0

We are using file streaming from the node server and we are also streaming the response to a file

If we loose connection mid download the module gets stuck and never finishes.
We are running a synchronous application and there for it stops working when the request module gets stuck.

I wrote a test code which you should be able to use to test this.

//for server install restify and throttle
npm i restify
npm i throttle
//for client install request
npm i request

server code

    var restify = require("restify");
    var throttle = require("throttle");
    var fs = require("fs");
    var server = restify.createServer();

    server.use(restify.bodyParser());
    server.use(restify.queryParser());

    server.get("/server", function(req,res){
      res.send(200, "This is the correct server");
    })

    server.get("/download", function(req,res){
      var trott = new throttle(512);
      fs.createReadStream("fileUpload/file.dat").pipe(trott).pipe(res);
    });

    server.listen(3000, function (){
      console.log("Download Server listening");
    });

client code

    var request = require("request");
    var fs = require("fs");
   // require("request-debug")(request);
    var reqObj = {
      url: "http://{url}:3000/download",
      headers: {}
    };

    request.get(reqObj).on("response", function(response){
      if(response.statusCode === 200){
        console.log("got file");
        var output = fs.createWriteStream("fileDownload/file.dat");
        response.pipe(output);
        output.on('finish', function(){          
          console.log("finished writing file");
        });

        output.on("error", function(error){
          console.log("write error: ", error.message);
        });   
      }
    }).on("error", function(error){
      console.log("request error: ", error.message);
    });

Request-Debug output

{ request: 
   { debugId: 1,
     uri: 'http://{host}:3000/download',
     method: 'GET',
     headers: { host: '{host}:3000' } } }
{ response: 
   { debugId: 1,
     headers: 
      { date: 'Fri, 15 Jul 2016 14:24:36 GMT',
        connection: 'close',
        'transfer-encoding': 'chunked' },
     statusCode: 200 } }

How to recreate

Deploy the server code to a remote server and create a file of size x

dd if=/dev/zero of=file.dat bs=1M count=250 //250M size

Run the client code on your desktop and turn off your internet adapted for more than 10 minutes. (I used a ubuntu virtual machine and turned off the 'Enable Networking')
When internet connection is resumed the module gets stuck and never finished the pipe and therefor the code in the output.on("finish", function is never run. (There the callback is called in our application)

If the internet connection is dropped for less that 10 minutes the output finish function is called but does not finish download the file. This scenario is okay for me because this will initiate a redownload.

The deadlock we have when the output finish function is never called is a big problem for us.

@stale
Copy link

stale bot commented Nov 23, 2018

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Nov 23, 2018
@stale stale bot closed this as completed Nov 30, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant