page.waitForNavigation hangs forever

[ecolss]

Steps to reproduce

Tell us about your environment:

• Puppeteer version: puppeteer@2.0.0
• Platform / OS version: MacOS Catalina, 10.15.3 (19D76)
• URLs (if applicable): https://www.escentual.com/catalogsearch/result/?q=face
• Node.js version: v12.13.0

What steps will reproduce the problem?
Full code as follows, just trying to go to 10 pages by clicking the next link on each page.

const pptr = require('puppeteer');

async function main() {
  const br = await pptr.launch({headless: false, defaultViewport: null});
  const page = await br.newPage();

  await page.goto(
    'https://www.escentual.com/catalogsearch/result/?q=face',
    {waitUntil: 'networkidle2'}
  );

  for (var i=0; i<10; ++i) {
    await Promise.all([
      page.waitForNavigation({waitUntil: 'load', timeout: 100000}),
      page.click('#adj-nav-container > div.category-products > div.toolbar > div > div.amount > div > ol > li.next > a'),
    ]);
    console.log(`waited, url=${page.url()}`);
  }
  await br.close();
}

main();

What is the expected result?
I was expecting the above code could go to 10 pages by clicking the next link on each page.

What happens instead?
However, the above code gets stuck at 2nd page forever, waiting waitForNavigation to finish forever.

[ecolss]

If I remove or replace the page.waitForNavigation in the above code, it works.
So I thought maybe there is no navigation event happening at all after the click?

But this speculation doesn't make sense, since according to the waitForNavigation documentation, if the page navigates to a new url, it should have a navigation event, right?

[dariomx]

I just got the exact same issue. After trying all sort of options for the waitForNavigation (timeouts, network idle, etc); I found your post and tried the same trick of deleting the waitForNavitation ... and indeed it worked for me as well.

Either we do not understand under what circumstances a navigation event is generated, or there is a bug somewhere that does not always generate that.

Thanks for the workaround though, hope this issue gets attended.

[bland328]

For what it's worth, I often run into this problem using both Puppeteer and the "Pyppeteer" Python port, and still have not figured out the "one true cure." Any insight as to the potential root cause of this would be greatly appreciated.

[lesagi]

hi, any solution to this problem? it happens to me as well... is there a way to overcome this issue?
I assume waiting for 'load' and 'contentload' under some considerations won't work as it reacts to a fired event that doesn't necessarily happen if the page is already loaded. but what about networkidle?

Thanks

[mitchied]

Any updates on this one?

[ushuz]

Either we do not understand under what circumstances a navigation event is generated, or there is a bug somewhere that does not always generate that.

I think it's a bug. Even if there's no navigation event, the wait should throw timeout error instead of hanging forever.

[ushuz]

Either we do not understand under what circumstances a navigation event is generated, or there is a bug somewhere that does not always generate that.

I think it's a bug. Even if there's no navigation event, the wait should throw timeout error instead of hanging forever.

[alecsci]

Reproduced it with puppeteer: 10.0.0, node: 14.16.0, npm: 6.14.11.
I also tried the recommended best practice from the official-docs and still hangs forever.
Similar behavior for page.waitForResponse

[stale]

We're marking this issue as unconfirmed because it has not had recent activity and we weren't able to confirm it yet. It will be closed if no further activity occurs within the next 30 days.

[stale]

We are closing this issue. If the issue still persists in the latest version of Puppeteer, please reopen the issue and update the description. We will try our best to accomodate it!