Baseline latency to convert html to pdf seems high compared to doing it directly with Chromium

[rreynier]

It seems that bootstrapping an html to pdf conversion may have a ton of overhead when going through Gotenberg. I stood up an instance in Docker and ran the following two commands from within the container. To note, test.html simply has "hello world" inside.

Directly via chromium: 0.137s

chromium --no-sandbox --headless --print-to-pdf=test.pdf test.html

Through Gotenberg API: 2.193s

curl \
--request POST 'http://localhost:3000/forms/chromium/convert/html' \
--form 'files=@"test.html"' \
-o test.pdf

Is this to be expected? I recognize I have to go through HTTP and whatever other layers exist, but 2 seconds seems like a lot just for baseline latency.

[gulien]

Good point! I have to monitor what’s taking so much time.

[gulien]

Alright, so I do remember the reason. In Gotenberg, I'm waiting for a bunch of events to be fired by Chromium: DomContent, Load, NetworkIdle, LoadingFinished, etc.

The reason is to make sure that all content of the page will appear in the PDF. For very simple HTML content, it is kind of overkill, but for most scenarios I do think it's the best.

That being said, I could find a way to offer some "performance mode" for simple HTML content.

[rreynier]

Hmm interesting. What is the risk presented here? I suppose when I run the command line im not waiting for any of these things and it seems fine (ive also thrown larger html documents at it before as well).

My usecase is to generate 1000's of small pdfs with some raw html and the 2sec overhead adds up quick.

[gulien]

As I said, it is just to make sure all the content is actually correctly loaded. It can be pictures, style from CSS files, Google fonts, etc.

In your case, you could spin up X Gotenberg instances thanks to your Docker orchestrator to parallelized your conversions.

[rreynier]

I have definitely considered the parallelized approach, but it introduces some other technical challenges in my project.

That said, even simply generating a small document direct to the end user, having 2000-3000ms baseline is still something I like to avoid. Generally speaking any endpoint I have I prefer to be under 500ms (ideally).

Either way, thanks for the intel. The other option I have is to install Chromium directly on the server and call the cli for it, losing some of the obvious benefits and features that Gotenberg provides!

[rreynier]

I know Gotenberg isn't pupeteer based, but after poking around pupeteer a bit, I can reproduce this issue with it as well. The issue seems to be that networkIdle takes ~2 seconds no matter what. I can pass a raw html document ("hello world") with no JS calls or no external calls and it still takes 2 seconds before the network idle event is given (networkidle0 or networkidle2 produce the same result). Listening for load or domcontentloaded completely removes any latency. Obviously this could be an issue with external resources, but if you are trying to build a blazing fast pdf converter, you may just base64 any images at which point load or domcontentloaded would be sufficient.

Unfortunately I have no Go experience so providing a PR with something like a "performance" flag is out of my scope of ability, but it certainly may be useful for lots of folks who are simply trying to create basic pdfs from html (invoices, reports, etc).

[gulien]

Gotenberg is using the same protocol as puppeteer 👍

Good to know, I could offer a « performance mode » that skip the network idle event.

[gulien]

I’ll try to come up with something for Gotenberg 8 (an edge tag will be available before the official release). I’ll let you know!

[rreynier]

Ok awesome, thank you!

It does look like there are 4 lifecycle events that are typically used for this configuration: https://pptr.dev/api/puppeteer.puppeteerlifecycleevent

It does beg the question, why are there ANY network connections even when I load html that has literally nothing besides "hello world". Maybe chromium is making some calls home and those are being considered active network connections?

[gulien]

I think the networkIdle0 and networkIdle2 are heuristics (i.e., fitting most use cases).

[gulien]

@rreynier gotenberg/gotenberg:edge now provides* a skipNetworkIdleEvent form field (bool, default to false) for the Chromium routes.

*check https://github.com/gotenberg/gotenberg/actions/runs/7185235602.

[rreynier]

Awesome! I will be testing this hopefully tonight or tomorrow. Thanks for jumping on this so quickly.

[rreynier]

Just tested this and I am seeing massive improvements in render speed. Locally I am seeing average render time go from around 2000-2500ms to a consistent 70ms. I have not spent much time confirming output though, but will report back.

[gulien]

Great! In my tests, it seemed identical with skipNetworkIdleEvent set to true or false (using "complex" HTML like here). I think in most cases, the assets are all loaded before the networkIdle event. One would argue that having skipNetworkIdleEvent set to true by default may be more suitable, but I prefer to make sure that in most scenarios all the content is rendered by default. I guess having a shiny "Optimize for Speed" chapter in the documentation may be enough lol

[rreynier]

I am surprised you are not able to reproduce what I am seeing. Your "complex" HTML examples are not far off from what im using.

Either way, it def makes a significant difference for me.

Thanks again!

[gulien]

Sorry, I meant that the PDFs seemed identical between the two scenarios, but the average response time is indeed way better with skipNetworkIdleEvent set to true.