High CPU usage

[benonymus]

Hey,
I recently switched to this library for pdf generation from wkhtmltopdf, and the cpu usage went up from 5-10% generally to 60-80+%.

Did I miss something?

Any tips?

[andreasknoepfle]

Hey @benonymus,
I suspect that the high CPU usage that you are seeing is due to Chromic PDF starting a Chrome instance with some tabs ready for PDF generation. This is great for a production server, so the PDF generation can happen very fast while keeping the same memory footprint, since the chrome tabs that are used to print are recycled.
You can also turn off this behavior by using on_demand mode. In this case there won't be any chrome instances started and kept around when you start the application, but only on demand when you generate a PDF (similar to what wkhtmltopdf will do).

I hope this already helps to bring some light into the case.

Cheers,
Andi

[benonymus]

Hey @andreasknoepfle,

Thanks for the quick reply!
Is this also something that could be tweaked by these settings:

defp chromic_pdf_opts do
  [
    session_pool: [size: 3]
    ghostscript_pool: [size: 10]
  ]
end

?
Or is that for different things?

[andreasknoepfle]

Yes, the session_pool flag is the right one to tweak how many browser tabs will be opened. However, I think the main CPU load comes form starting chrome itself, which only happens once. However this explains only a short term raise in CPU load when the system starts. A chrome process that is just open and waiting for PDF generation to happen should not eat up much CPU. In case you experience this load continuously there is probably something else going on.

Could you run htop or something while you generate your PDFs, so we can see which process consumes the CPU (if it is the Erlang process or Chrome itself). Also could you give some context on what exactly you are printing? If it is a page with JavaScript running on page load, this might also be an explanation...

The ghostscript_pool configuration however will not affect any of this, as long as you do not convert you PDF to PDF-A since there Chromic PDF does not have open ghostscript sessions or anything like that. This config is just for having an upper limit of maximum ghostscrpt executions that will run in parallel.

[benonymus]

Hey,
Thanks again of the detailed answer!

The load is consistently on that level, and I am pretty sure we are not generating that many pdfs.
The templates have no js in them.
I will make some test, it could be something else as well, just this was my first suspect.
I will update the issue with my findings!

[benonymus]

Hey @andreasknoepfle is there no default session_pool?
I just checked the logs, I have this error:

** (Mix) Could not start application xxxx: XXXX.Application.start(:normal, []) returned an error: shutdown: failed to start child: ChromicPDF
2021-02-05T15:48:34.558+01:00 ** (EXIT) shutdown: failed to start child: ChromicPDF.Browser
2021-02-05T15:48:34.558+01:00 ** (EXIT) shutdown: failed to start child: ChromicPDF.Browser.SessionPool
2021-02-05T15:48:34.558+01:00 ** (EXIT) an exception was raised:
2021-02-05T15:48:34.558+01:00 ** (ArgumentError) pool_size must be more than 0, got: 0
2021-02-05T15:48:34.558+01:00 (nimble_pool 0.2.3) lib/nimble_pool.ex:219: NimblePool.start_link/1
2021-02-05T15:48:34.558+01:00 (stdlib 3.12.1) supervisor.erl:379: :supervisor.do_start_child_i/3

But locally I am not getting any errors

[maltoe]

hey @benonymus

The error reads indicates you have session_pool: [size: 0] in your options which is not allowed. As @andreasknoepfle said, if you want to disable the session pool, please consult the docs for the "on demand" mode. This will effectively disable the background Chrome process and spawn it when needed.

However, I don't think the session pool, or the background Chrome process is the reason for your high CPU load - or if it is, I'd really like to understand why, as so far I've never heard of any CPU load issues caused by an idling Chrome process. As long as you don't use it (as in, print PDFs) this process is just sitting there and doing nothing.

To be able to debug your issue further, it would be really helpful if you provided us with more information. htop as @andreasknoepfle suggested to see which OS process is actually eating your CPU, then environment information (esp. Chrome version, but also OS, deployment environment, etc.)

[benonymus]

Hey @maltoe

I think the high cpu usage is just a symptom, the compilation keeps failing and that cause the high cpu.

But I am not setting session_pool in my settings, this 0 I think somehow comes from the default calculation which would mean there are 0 schedulers online.

I will try to set my session_pool and report back.

[maltoe]

But I am not setting session_pool in my settings, this 0 I think somehow comes from the default calculation which would mean there are 0 schedulers online.

uh? interesting that during compilation you have 0 schedulers, how does it even compile anything? But that's actually a good point, it should get the number of schedulers at runtime. Will add a fix for that.

[maltoe]

Is the compilation failing due to ChromicPDF though?

[benonymus]

@maltoe Yes it looks like everything goes fine and we fail with the error I sent earlier.

[benonymus]

You can get the same error locally if you hard code 0 as size for the session_pool

[maltoe]

👍

Just found out (slight facepalm here):

div(System.schedulers_online(), 2)

If you only have 1 scheduler online (usually this means = 1 core), this will return 0.

[maltoe]

#102

[benonymus]

@maltoe Good catch! I think then it is that, we have a tiny vm.

[benonymus]

@maltoe Also so the compilation passes, we fail when the application tries to start as the error shows, but not important anymore.

[maltoe]

awesome!

So and when you set the option to some value > 0 it works? Or did you say you were going to try that out another day?

[session_pool: [size: 1]]

[benonymus]

Waiting for the deployment at the moment

[maltoe]

You can get the same error locally if you hard code 0 as size for the session_pool

makes sense, I'll see if I can add a validation so people don't set that accidentally and run into the issues you were seeing.

[edit] 🤔 hmm actually the exception from NimblePool might be enough

[benonymus]

I was also wondering when you use on_demand mode, shouldn't that ignore the pool_size, or how does that interaction work?
If you set 0 and do on_demand: true you still get the same error

[maltoe]

🤔

shouldn't that ignore the pool_size

It should and it does, and it works for me.. ?

iex(1)> ChromicPDF.start_link(on_demand: true, session_pool: [size: 0])
{:ok, #PID<0.248.0>}
iex(2)> ChromicPDF.print_to_pdf({:html, "test"})
{:ok,

[benonymus]

Oh right, I guess I had it mixed up with false on_demand, sorry 🥇

So if I use on_demand the pool size is always one?

Edit yes it is I see

[maltoe]

yes, it is because the pool size becomes kind of irrelevant as the "on demand" Chrome process is only used for a single operation and then killed again. It's really only a development feature as we kept having a bunch of zombies on our dev machines, it's pretty tricky to kill external processes when the BEAM is shutdown, especially with abort from iex. Not so much an issue on production, of course.

[benonymus]

Yeah, that is why I was not too keen to use that option on prod if I didn't have to, plus the potential performance penalty...

[benonymus]

I will give an update once the release(not up to me) is out, but I am pretty confident it will work.
And then down the line just update the lib once your fix is merged, and rely on that.

Thanks for all the help, very cool library!

[maltoe]

Thanks for pointing me to this issue! Hope it'll work for you.

And @andreasknoepfle is going to get sooo many emails 😄

[benonymus]

Well only putting it in on_demand mode fixed the problem, seemed like just changing the pool size didn't fix it, but I think it was in some bad state, kept retrying.
Once the fix is out I will give it a go, and stop using on_demand mode.

[maltoe]

seemed like just changing the pool size didn't fix it

🤔 didn't fix it = you still had high CPU load?

Released 0.7.1 now, pls give it a try.

[benonymus]

So there was high CPU load because it was stuck in this recompile loop, and that manifested itself in the high cpu usage.
Somehow even with the pool size it was stuck on 0, but that could have been something else with it being stuck.
The on_demand mode did go through.
I will give the update a go at some point and let you know!

[maltoe]

@benonymus closing this for now, feel free to re-open if you still experience issues