Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Requests with timeout still hand indefinitely #124

Closed
MillionIntegrals opened this issue Oct 7, 2015 · 14 comments

Comments

@MillionIntegrals
Copy link

commented Oct 7, 2015

I have w project where I make tens of thousands of requests to a backend service, scraping some API. I don't control the backend. To make the process fast, I post the requests in chunks, in parallel.

Sometimes, though, the backend just hangs indefinitely on a request. For that I'd like to have a timeout and just resend new request. Here is the code that I use for that purpose (I'm using the ES7 async/await functionality):

while(true) {
    try {
        let response = await axios.get(url, {timeout: timeout});
        return response.data;
    } catch (e) {
        if (!silent) {
            console.log(`Exception caught: ${JSON.stringify(e)}`);
        }
    }
}

The problem is, even with the timeout settings some of the requests above hang indefinitely - by average 1 per 3000.

To check if the problem is with my code, or with the library I've implemented similar functionality using superagent:

while(true) {
    try {
        let promise = new Promise((resolve, reject) => {
            superagent.get(url).timeout(timeout).end((error, response) => {
                if (error) {
                    reject(error);
                } else {
                    resolve(response);
                }
            })
        });
        let response = await promise;
        return response.body;
    } catch (e) {
        if (!silent) {
            console.log(`Exception caught: ${JSON.stringify(e)}`);
        }
    }
}

And the above code works as expected.

@MillionIntegrals

This comment has been minimized.

Copy link
Author

commented Oct 7, 2015

I did some digging in the code, and from what I see is that axios is handling request timeout by issuing the req.abort() function (https://github.com/mzabriskie/axios/blob/master/lib/adapters/http.js#L92):

  // Handle request timeout
  req.setTimeout(config.timeout, function () {
    req.abort();
  });

While superagent is doing a bit more (https://github.com/visionmedia/superagent/blob/5a320d7d3222935718bfde91cae91ade4b371826/lib/node/index.js#L833):

  if (timeout && !this._timer) {
    debug('timeout %sms %s %s', timeout, this.method, this.url);
    this._timer = setTimeout(function(){
      var err = new Error('timeout of ' + timeout + 'ms exceeded');
      err.timeout = timeout;
      err.code = 'ECONNABORTED';
      self.abort();
      self.callback(err);
    }, timeout);
  }

Both aborting the request and fulfilling the callback at the same time.

@mzabriskie

This comment has been minimized.

Copy link
Member

commented Oct 7, 2015

IIRC aborting the request should put it into the on error handler. I'll check again to verify.

@WoLfulus

This comment has been minimized.

Copy link

commented Nov 22, 2015

This is also happening for me.

@Elyx0

This comment has been minimized.

Copy link

commented Apr 22, 2016

@mzabriskie This is still happening at least in React-Native

axios.post(`/create/`, Qs.stringify(payload),{timeout: 1000})
    .then(json => {}).catch(e => {}).

My request is sent, the result is there before timeout runs out and the .then() is not called if I add the timeout parameter.

@amithgc

This comment has been minimized.

Copy link

commented Jan 27, 2017

any update on this issue? I have the same problem in some of the android devices (React Native), Timeout doesn't happen at all.

@mattb-prg

This comment has been minimized.

Copy link

commented Apr 13, 2017

@amithgc still having this issue. running on android too(emulator)

@frevib

This comment has been minimized.

Copy link

commented Aug 24, 2017

This workaround works (for React-Native):
#647 (comment)

@milewski

This comment has been minimized.

Copy link

commented Jan 7, 2018

the same is happening for me... after a bunch of request... one will hang forever... :( ... using nodejs

@1c7

This comment has been minimized.

Copy link

commented Jan 23, 2018

happening to me too, using React Native 0.52 for Android

@kdomagal

This comment has been minimized.

Copy link

commented Feb 18, 2018

happens in nodejs as well. Why is this issue closed @mzabriskie ?

@emilyemorehouse

This comment has been minimized.

Copy link
Member

commented Feb 18, 2018

From what I can gather, the original issue is solved. Timeouts are working properly for me:

screen shot 2018-02-18 at 10 58 40 am

Is this just an issue when many requests are being sent? If someone can provide code to reproduce the issue, I'm happy to look into it.

@kdomagal

This comment has been minimized.

Copy link

commented Feb 18, 2018

ok actually issue is not the timeout I think. Simply try to do couple of hundreds of requests in a loop. It will hang after few.

@emilyemorehouse

This comment has been minimized.

Copy link
Member

commented Feb 18, 2018

Hm. I tried with 1000 requests and it didn't trip it up for me. 😞

@ksnyde

This comment has been minimized.

Copy link

commented Sep 4, 2018

Using the following code -- which overly verbose in error handling -- I'd expect there be no way to not get a message to stdout from the Axios get call but indeed at random intervals it fails and just hangs. I rerun from where I left off and the endpoint which failed almost always works only to have it fail randomly somewhere else. I would strongly support this being a real issue.

try {
    content = await axios.create().get<string>(url, {timeout: 5000});
    console.log(chalk.grey('- axios content returned from endpoint'));
    
    if(content.status !== 200) {
      console.log(chalk.red(`- Returned status from ${url} was ${chalk.bold.yellow(String(content.status))}}`));
      problemsCaching.push(business);
      return business;
    }
    if(!content.data) {
      console.log(chalk.red(`- Axios responded correctly but payload is empty (${url})`));
      problemsCaching.push(business);
      return business;
    }
  } catch (e) {
    console.log(chalk.red(`- Problems loading "${url}" for ${business.name}`));
    problemsCaching.push(business);
    return business;
  }
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.