New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use a pool of browsers to execute render requests #66
Comments
Nice find! That looks like it would do pretty much exactly what we need, and the |
@DanCech Looking at puppeteer/puppeteer#4949 introducing |
It looks like we could let puppeteer install chromium itself rather than pre-installing from the alpine edge package, which in theory at least should be more reliable since it's going to download the same version the release was tested against. If we go that route and update to 2.0.0 then the per-page timezone should hopefully Just Work. |
@DanCech problem though is that the puppeteer cannot automatically download chromium for alpine, at least that didn't work before. But one alternative would be to add an ubuntu image as well and/or build a custom one. |
Aha, that is a problem then, since the alpine packaged version in edge right now is 78.0.3904.108-r0 https://pkgs.alpinelinux.org/package/edge/community/x86_64/chromium which seems to be the current stable release as that's the version in the ubuntu etc repos also. |
Looks like alpine now has 79.0.3945.88-r0 in 3.11 and in edge |
Adds clustered browser using puppeteer-cluster lib. Adds reusable browser only launching one browser at start. In case of browser crash it will currently not be restarted. Adds browser timings metrics to be able to monitor how long each individual step takes, like launching browser, new page, navigate, wait for panels to render and take screenshot. Adds browser info metric showing browser version, like HeadlessChrome/79.0.3945.0. Support for configuring browser mode (default, clustered, reusable) and clustering mode (browser, context) and max concurrency. Environment variables also supported. Refactor and restructure files/code. Closes #66
Currently a new chrome instance is launched for every render request, which introduces latency and the possibility of running out of memory if too many requests come in at once.
Originally this requirement was because there wasn't a way to set the timezone for a particular page in puppeteer, but that is now available: https://github.com/puppeteer/puppeteer/blob/v2.0.0/docs/api.md#pageemulatetimezonetimezoneid
I'd suggest using https://github.com/Vincit/tarn.js to manage the pool of puppeteer browser objects so that we can easily handle multiple requests by running up to
pool.max
chrome instances and queuing requests until an instance is available.The text was updated successfully, but these errors were encountered: