Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to page.goto() to retry on ERR_CONNECTION_REFUSED #2460

Closed
melissachang opened this issue Apr 26, 2018 · 7 comments

Comments

Projects
None yet
6 participants
@melissachang
Copy link
Contributor

commented Apr 26, 2018

I start servers and run Puppeteer tests at roughly the same time. When tests first attempt page.goto(), it fails with ERR_CONNECTION_REFUSED because server hasn't finished initializing. So I wrote manual retries for page.goto().

It would be nice if there was an option to page.goto() to implement the retry logic itself. This would be the equivalent of hitting refresh in the browser, until page loads.

I would think this would be a common request, but it appears it's not. I'm interested to hear how people automate their deployment such that server initialization is guaranteed to be done before Puppeteer tests are run.

@benjamingr

This comment has been minimized.

Copy link

commented Apr 26, 2018

I don't think puppeteer needs to solve this general problem of retrying actions.

I recommend you use a library that does retries.

Let's say you do the following which fails:

await page.goto('http://returns-errors-sometimes.example.com');

Here is a simple retry, there are modules on npm that do this with exponential backoff or a maximal number of attempts:

const retry = (fn) => fn().catch(retry.bind(fn));

Which would let you do the following instead which would retry page.goto until it succeeds:

await retry(() => page.goto('http://returns-errors-sometimes.example.com'));
@melissachang

This comment has been minimized.

Copy link
Contributor Author

commented Apr 26, 2018

Your minimal example isn't realistic because it doesn't sleep. But point taken, I'm sure there are npm packages that would let me do it in fewer lines of code.

There are two separates ideas.

  1. Many users of puppeteer have a need to retry page.goto
  2. Users of an API should implement retry themselves, vs the API doing it for them

I have no idea about 1.

Regarding 2, as a general software principle, if enough users need retry, I believe that implementing lower down in the stack (at API level) is better. Not everyone agrees with this.

Edit: I guess 2 is less relevant in this case. 2 is more for when a flaky error occurs. In this case, page.goto() is supposed to fail.

@curran

This comment has been minimized.

Copy link

commented Jul 6, 2018

Retying outside the Puppeteer API makes sense. Looks like not required as part of Puppeteer.

Here's the solution I'm using:

const retry = (fn, ms) => new Promise(resolve => { 
  fn()
    .then(resolve)
    .catch(() => {
      setTimeout(() => {
        console.log('retrying...');
        retry(fn, ms).then(resolve);
      }, ms);
    })
});

Here's the code that calls it (inspired by UI testing with Puppeteer and Mocha):

it('should open page', async () => {
  browser = await puppeteer.launch({ headless: false });
  page = await browser.newPage();
  const response = await retry(() => page.goto('http://localhost:3000'), 1000);
  assert.equal(response.status(), 200);
});

This works well for waiting for the server to spin up. Just like I would do when manually testing. Refresh, refresh, refresh, boom!

@aslushnikov

This comment has been minimized.

Copy link
Collaborator

commented Jul 19, 2018

It would be nice if there was an option to page.goto() to implement the retry logic itself. This would be the equivalent of hitting refresh in the browser, until page loads.

I don't think puppeteer should do any magic and run retries. Closing given there's no demand for this.

@endel

This comment has been minimized.

Copy link

commented Aug 31, 2018

Hey @benjamingr, do you mind sharing the NPM modules you've mentioned here?

Here is a simple retry, there are modules on npm that do this with exponential backoff or a maximal number of attempts:

I've made a quick search on NPM and couldn't find them. Cheers!

@benjamingr

This comment has been minimized.

Copy link

commented Aug 31, 2018

@endel sure: https://www.npmjs.com/package/promise-retry for example.

It's essentially

const delay = util.promisify(setTimeout);
const retry = (fn, retryDelay = 100, numRetries = 3) => async (...args) => 
  for (let i = 0; i < numRetries; i++) {
    try {
      return await fn(...args); 
    } catch (e) {
      if (i === numRetries - 1) throw e;
      await delay(retryDelay);
      retryDelay = retryDelay * 2;
    }
  }
};
@Xample

This comment has been minimized.

Copy link

commented Jan 13, 2019

@curran nice snippet! for people trying to type it in TS:

// retry.ts
export const retry: <T>(fn: () => Promise<T>, ms: number) => Promise<T> = <T>(fn: () => Promise<T>, ms: number) => new Promise<T>((resolve) => {
    fn()
        .then(resolve)
        .catch(() => {
            setTimeout(() => {
                console.log('retrying...');
                retry(fn, ms).then(resolve);
            }, ms);
        });
});

Just because it was cumbersome to do it…

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.