-
Notifications
You must be signed in to change notification settings - Fork 99
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Best way to optimize using Grover (just discovered 750 chrome processes opened) #102
Comments
Not sure what you've got going on there. You're using Chrome rather than Chromium? It's possible there is some sort of memory leaking happening; either something in what you're converting or something in the version of Chrome you're using? Our servers get rolled a number of times per week, but they process decent numbers of PDFs don't have any signs of memory or process bloat.
More than likely this isn't something I can help with. Grover really is just a thin wrapper around Puppeteer (using NodeJS), which of course is what makes the calls out to Chromium/Chrome/Firefox. I would suggest starting by looking at what versions of each of these tools you're using. If you're all up to date, then suggest taking this issue upstream. |
Gotcha. Thanks @abrom. I think it's actually using Chromium from what I can see. I found this in htop: When running the same command from the CLI, I can see it's running Chromium 83.0.4103.0:
Think it's most likely related to Chromium in this case? I'm not too knowledgeable with node modules but it looks like it's just loading whichever chromium version is included in the puppeteer module? Not quite sure if that's an accurate statement or not. EDIT Tried updating Chromium and ended up with version 90.0.4396.0 and the issue is still happening even after using |
@abrom are your servers using Grover in a docker container? Wondering if this may be the common denominator. I've confirmed in 3 docker containers that the issue happens and on 2 non-docker instances that it doesn't happen. |
Hi @altjx, no we don't use docker for the instances generating PDFs. Sounds like that might be the smoking gun. This might help?? https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md#running-puppeteer-in-docker This seems like a possible culprit:
They also suggest to include the Seems likely it's one (or many) of the above.. |
Gotcha. Thanks so much for that tip. First time running into this so this is extremely helpful for me! |
@altjx Apologies for being off-topic, but any chance you could share how you got Puppeteer to pick up your stylesheets while containerized? I've got Grover emitting correctly prefixed URLs but am having no luck getting my styles to appear in rendered PDFs. Debugging's a bit trickier when you have to be headless, too. Edit: Ooh, never mind. Pro-tip to anyone else doing this: if you're running from a rake task or something similar using |
Haha no problem! Glad you got it figured it out. Feel free to PM me if you run into any other situations. We're relying on this gem pretty heavily so it's pretty critical for me. |
@jandresrodriguez I provided a possible list of reasons and possible solutions above. I don't use Grover with Docker in our production environment so I have not seen this issue myself. |
Adding init to the docker-compose worked for me! 🎉 |
I'm going to close this as there hasn't been any movement here for a while. I will suggest that the browser_ws_endpoint option might be of use in this case though. It allows Grover to connect to a remote Chromium instance, but also doesn't close the browser, only the page, which might provide a performance bump |
Hi there,
I am having a similar issue to #60, but my scenario is just slightly different. I am using Sidekiq (plus Ruby on Rails) to generate a PDF document within a docker container. I am using Sidekiq worker to basically process HTML content (which sometimes have up to two charts).
Here's how I'm using them:
I realized that, lately, the Sidekiq worker has been failing with these types of errors:
The above output is the most common. The one below is one I've just started seeing:
I haven't seen the error above before except for today. I did some investigating and, from what I can see, each time I run something as simple as:
This spawns 5 processes that contains "chrome" in it and they don't seem to ever close.
I wonder if this is my problem? If so, is there a best practice that I should be implementing to ensure that the chrome process closes once it's finished? As you can see from the above example, it's just loading a very basic HTML with small data, but yet the 5 chrome processes stay opened.
It gets to the point to where I can't even kill the chrome process with
kill -9 [pid]
:The text was updated successfully, but these errors were encountered: