New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

slow performance with Pygments? #483

Closed
dmalan opened this Issue Jul 4, 2017 · 4 comments

Comments

Projects
None yet
2 participants
@dmalan

dmalan commented Jul 4, 2017

Is particularly slow performance on HTML generated by Pygments to be expected, even on relatively small files? On Ubuntu 14.04 with WeasyPrint 0.39 (and, to be fair, an older Cairo and Pango), we're currently seeing the following if we try to syntax-highlight, e.g., weasyprint/fonts.py (which only has 294 lines) from WeasyPrint's own package:

$  pygmentize -O full -o fonts.html fonts.py
$ time weasyprint fonts.html fonts.pdf
/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/weasyprint/text.py:29: UserWarning: There are known rendering problems with Cairo <= 1.14.0
  warnings.warn('There are known rendering problems with Cairo <= 1.14.0')
/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/weasyprint/fonts.py:46: UserWarning: @font-face support needs Pango >= 1.38
  warnings.warn('@font-face support needs Pango >= 1.38')

real	0m46.330s
user	0m45.970s
sys	0m0.200s

But is 46 seconds (inside a Docker container on a recent Mac Pro) indeed to be expected for a 294-line syntax-highlighted file? We've experimented with a few Weasy and Pygments options, but none seem to boost performance noticeably. Runtime seems to increase noticeably, though, as soon as the input fills more than 1 page of the PDF.

Sample inputs at https://gist.github.com/dmalan/9519e730b207e524bc6165966a492acf, and sample output at https://github.com/Kozea/WeasyPrint/files/1120994/fonts.pdf.

Runtime aside, everything is rendering quite nicely. Thank you!

@liZe liZe added the performance label Jul 5, 2017

@liZe

This comment has been minimized.

Show comment
Hide comment
@liZe

liZe Jul 6, 2017

Member

Re-applying the "optimization" removed in cff4452 gives me -30% rendering time for your use case. We should try to revert this commit in a smart way. And I'm sure that we can do even better than that.

Member

liZe commented Jul 6, 2017

Re-applying the "optimization" removed in cff4452 gives me -30% rendering time for your use case. We should try to revert this commit in a smart way. And I'm sure that we can do even better than that.

@liZe

This comment has been minimized.

Show comment
Hide comment
@liZe

liZe Jul 6, 2017

Member

And I'm sure that we can do even better than that.

Of course, we can do better than that! When there's no text wrap, we already know where to break lines: at \n characters…

Member

liZe commented Jul 6, 2017

And I'm sure that we can do even better than that.

Of course, we can do better than that! When there's no text wrap, we already know where to break lines: at \n characters…

@liZe liZe added this to the v0.40 milestone Jul 7, 2017

@liZe liZe closed this in 46da8b5 Jul 7, 2017

@liZe

This comment has been minimized.

Show comment
Hide comment
@liZe

liZe Jul 7, 2017

Member

It now takes less than 2s instead of about 28s for me. Not that bad!

For the record: cutting the lines isn't that useful in this case because there are lots of spans.

Member

liZe commented Jul 7, 2017

It now takes less than 2s instead of about 28s for me. Not that bad!

For the record: cutting the lines isn't that useful in this case because there are lots of spans.

@dmalan

This comment has been minimized.

Show comment
Hide comment
@dmalan

dmalan Jul 7, 2017

Amazing, thanks so much for such a quick commit, Guillaume. Just went from 1m8s to 4s (on a laptop) on our end! Game-changing for sure. Much appreciated!

dmalan commented Jul 7, 2017

Amazing, thanks so much for such a quick commit, Guillaume. Just went from 1m8s to 4s (on a laptop) on our end! Game-changing for sure. Much appreciated!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment