🚀 Feature Request
The proposed feature involves enhancing the navigation methods in the Playwright library, specifically goto, goBack, goForward, etc., by adding a new "networkAlmostIdle" option to the "waitUntil" parameter.
Example
await page.goto('https://example.com', { waitUntil: 'networkAlmostIdle' });
Motivation
I am using Playwright as a crawler for web pages and I have encountered a need for a more nuanced waiting option during navigation. Currently, the "waitUntil" option in methods like goto, goBack, goForward, etc., supports "load", "domcontentloaded", "networkidle". However, I've found that "networkidle" is not always suitable for my use case because some pages are constantly loading new resources, which makes the crawler wait indefinitely.
On the other hand, options like "load" and "domcontentloaded" sometimes return too early, before all relevant resources are loaded. Therefore, I propose the addition of a new "networkAlmostIdle" option. This option would wait until network activity has almost ceased but not completely, allowing for some ongoing network requests. This would be particularly useful for crawling dynamic websites that continuously fetch resources but have periods of relative inactivity that can be used as signals for page readiness. This feature would provide a more flexible and efficient way to ensure that the page is correctly loaded before the crawler proceeds, improving the accuracy and reliability of the crawling process. I believe this would be a valuable addition to Playwright's navigation capabilities and would benefit not only my project but also other use cases where a balance between waiting for page readiness and avoiding indefinite waits is needed.
It's worth noting that this feature is already implemented in the Taiko library, which I have used in the past. The "networkAlmostIdle" event is also supported by the Chrome Remote Interface. Therefore, it would be beneficial to have this feature in Playwright as well, to provide a consistent and efficient way of handling page readiness across different libraries and tools.
🚀 Feature Request
The proposed feature involves enhancing the navigation methods in the Playwright library, specifically goto, goBack, goForward, etc., by adding a new "networkAlmostIdle" option to the "waitUntil" parameter.
Example
await page.goto('https://example.com', { waitUntil: 'networkAlmostIdle' });Motivation
I am using Playwright as a crawler for web pages and I have encountered a need for a more nuanced waiting option during navigation. Currently, the "waitUntil" option in methods like goto, goBack, goForward, etc., supports "load", "domcontentloaded", "networkidle". However, I've found that "networkidle" is not always suitable for my use case because some pages are constantly loading new resources, which makes the crawler wait indefinitely.
On the other hand, options like "load" and "domcontentloaded" sometimes return too early, before all relevant resources are loaded. Therefore, I propose the addition of a new "networkAlmostIdle" option. This option would wait until network activity has almost ceased but not completely, allowing for some ongoing network requests. This would be particularly useful for crawling dynamic websites that continuously fetch resources but have periods of relative inactivity that can be used as signals for page readiness. This feature would provide a more flexible and efficient way to ensure that the page is correctly loaded before the crawler proceeds, improving the accuracy and reliability of the crawling process. I believe this would be a valuable addition to Playwright's navigation capabilities and would benefit not only my project but also other use cases where a balance between waiting for page readiness and avoiding indefinite waits is needed.
It's worth noting that this feature is already implemented in the Taiko library, which I have used in the past. The "networkAlmostIdle" event is also supported by the Chrome Remote Interface. Therefore, it would be beneficial to have this feature in Playwright as well, to provide a consistent and efficient way of handling page readiness across different libraries and tools.