*** buffer overflow detected ***: wkhtmltopdf terminated

[fitterfly-fenil]

Hi,
While trying to generate large PDFs with more than 150 pages with header and footer I get an error - *** buffer overflow detected ***: wkhtmltopdf terminated. Could anyone let me know what could be the possible reason for this?

[ashkulz]

@fenilp: you have provided me absolutely zero information. At least provide:

• Version (wkhtmltopdf -V)
• OS/Platform
• sample HTML/CSS which reproduces this issue
• exact command line arguments used

and then reopen this issue.

[fitterfly-fenil]

@ashkulz : Hi Ashish, under an NDA with client I cant provide you the complete details but whatever I can is as below

Version - wkhtmltopdf 0.12.1-9615f00 (with patched qt)
OS/Platform - Linux/Ubuntu
Sample HTML - I cannot provide you the sample HTML under NDA with the client
Command Line - wkhtmltopdf --header-html https://www.cantdisclosedomain.com/pdf/pdf_header.php --footer-html https://www.cantdisclosedomain.com/pdf/pdf_footer.php https://www.cantdisclosedomain.com/sample.html sample.pdf

Hope this is sufficient. Let me know if you need anymore details.

Thanks,
Fenil

[fitterfly-fenil]

@ashkulz : I generated the PDF without headers and footers, it got generated. The size of the generated PDF is 135 MB and number of pages are 1270. I have set the ulimit to 5000.

[ashkulz]

@fenilp: can you try to generate some sample HTML which can reproduce the issue? I'm not interested in actual data, all I care is that the issue is reproducible.

[avoinea]

Same issue here.
PDF size: 226 pages
ulimit -n 4096
ulimit -u 4096

It works without header or footer, but it doesn't with both.

wkhtmltopdf --page-size A4 --page-offset 2 --print-media-type --encoding utf-8 --debug-javascript --margin-top 32 --margin-bottom 32 --margin-left 20 --margin-right 20 --no-outline --javascript-delay 200 toc --xsl-style-sheet toc.xls --disable-toc-links http://example.com/countries/pdf.body --load-error-handling ignore --header-html http://example.com/countries/pdf.header --header-font-size 9 --header-spacing 5 --header-font-name Verdana --footer-html http://example.com/countries/pdf.footer --footer-font-size 9 --footer-font-name Verdana --footer-spacing 5 out.pdf

[joshuapinter]

Having the exact same issue here.

I'm generating PDFs that are between 500 - 1000 pages in length, although very simple data tables. Including the header and footer causes this buffer overflow issues.

The original problem was "Too many open files." which was fixed by adjusting the nofile limit on the servers.

Any updates or suggestions with this, guys?

Many Thanks!

[ashkulz]

@joshuapinter: please post a reproducible test case in a gist or upload it somewhere.

[joshuapinter]

Thanks for the response, @ashkulz. I was able to work around the issue by doing two things:

1. I increased the open file limit and stack size limit of my servers by adding the following config to /etc/security/limits.conf:

   *    hard nofile 1000000
   *    soft nofile 1000000
   *    soft stack  81920
   root hard nofile 1000000 # Need these two lines because the wildcards (above) 
   root soft nofile 1000000 # are not applied to root automatically.
   root soft stack  81920

2. Instead of using a S3 image in the header, I store the image locally. Not doing this seemed to required more resources.

You're gonna have to play with the limits to see what works with the size of the PDFs you're generating but it's surprising how high these limits have to be to prevent issues. Ideally, we could refactor wkhtmltopdf to not use so many resources but this is a decent workaround.

[daniar23]

Unfortunately increasing stack size and open file limit didn't bring me anything. But then I found out that the problem was in setting up pathes of footer and header with http urls. So I've changed them to local pathes and it solved the problem.

[joshuapinter]

@daniar23 Can you give an example of what your header and footer paths looked like before and then after you changed them?