Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OSError: Too many open files #396

Closed
sander76 opened this issue Dec 2, 2016 · 9 comments
Closed

OSError: Too many open files #396

sander76 opened this issue Dec 2, 2016 · 9 comments
Assignees
Labels
bug
Milestone

Comments

@sander76
Copy link

@sander76 sander76 commented Dec 2, 2016

I am creating pdf's from a (static) site which is built every night.
But after some time weasyprints ends with the above OSError.
I have been trying to simplify the issue and have come up with the following python script:
(Running Python 3.5.2, Ubuntu 16.04.1 LTS (GNU/Linux 4.4.0-51-generic x86_64))

from weasyprint import HTML
import logging

logging.basicConfig(level=logging.DEBUG)

URL = "http://tools.hde.nl/menc/site/guides/Pliss%C3%A9%20%26%20Duette%C2%AE%20Bottom-Up%20programming/"
OUT = "/home/admin-s/test.pdf"

for lp in range(0, 300):
    try:
        HTML(URL).write_pdf(OUT)
    except OSError as e:
        logging.exception("**************** ERROR AT ATTEMPT: {} *******************".format(lp))
        break

In my case the error occurs after approx 60 loops (varies per situation).
Stacktrace:

ERROR:root:**************** ERROR AT ATTEMPT: 63 *******************
Traceback (most recent call last):
  File "tester.py", line 11, in <module>
  File "/home/admin-s/envs/weasy/lib/python3.5/site-packages/weasyprint/__init__.py", line 178, in write_pdf
  File "/home/admin-s/envs/weasy/lib/python3.5/site-packages/weasyprint/__init__.py", line 143, in render
  File "/home/admin-s/envs/weasy/lib/python3.5/site-packages/weasyprint/document.py", line 325, in _render
  File "/home/admin-s/envs/weasy/lib/python3.5/site-packages/weasyprint/css/__init__.py", line 712, in get_all_computed_styles
  File "/home/admin-s/envs/weasy/lib/python3.5/site-packages/weasyprint/css/__init__.py", line 162, in find_stylesheets
  File "/home/admin-s/envs/weasy/lib/python3.5/site-packages/weasyprint/__init__.py", line 261, in __init__
  File "/home/admin-s/envs/weasy/lib/python3.5/site-packages/weasyprint/css/__init__.py", line 692, in preprocess_stylesheet
  File "/home/admin-s/envs/weasy/lib/python3.5/site-packages/weasyprint/fonts.py", line 229, in add_font_face
OSError: [Errno 24] Too many open files: '/tmp/tmp1a03t715'
@liZe liZe added the bug label Dec 2, 2016
@sander76
Copy link
Author

@sander76 sander76 commented Dec 4, 2016

Doing a little digging around I found out the following:

ulimit -n shows the amount of allowed open files. It was set at 1024 which was pretty low compared to a docker instance I tried (where the issue was not occurring).

I edited /etc/security/limits.conf by adding

*               soft    nofile          65535
*               hard    nofile          65535

Now ulimit -n shows the number above and the issue like described in the first post is gone.
Don't know though whether this is a workaround for an actual weasyprint issue. But for now it works for me !

@liZe
Copy link
Member

@liZe liZe commented Dec 6, 2016

There's a real problem about this, temporary files used for fonts management are deleted but still open by the python process. That's why the 5th item in #376 is needed for example.

@sander76
Copy link
Author

@sander76 sander76 commented Dec 7, 2016

@liZe Alright. But the 5th item is windows related if I understand correctly. My issue is all linux...

@liZe
Copy link
Member

@liZe liZe commented Dec 7, 2016

Yes, the file is open on both Windows and Linux, but Windows doesn't accept deleting open files when Linux does. The file is marked as open+deleted on Linux, when it should be closed and deleted afterwards.

@sander76
Copy link
Author

@sander76 sander76 commented Dec 7, 2016

@liZe
Copy link
Member

@liZe liZe commented Dec 7, 2016

Obvious fact: this bug is not related at all to the Windows support 😄.

I really wonder if
anyone else has succeeded...

At least I did, and it works well.

There are lots of bugs related to the Windows support in the bug tracker, so I assume that it's working for some people too. There's almost no Windows-specific code in WeasyPrint, the only hard part is the installation. The documentation about that has been improved during the last months, you should give another try!

@liZe liZe self-assigned this Dec 7, 2016
@liZe liZe added this to the 0.34 milestone Dec 7, 2016
@liZe liZe closed this in bf6911d Dec 7, 2016
@liZe
Copy link
Member

@liZe liZe commented Dec 7, 2016

It's now fixed, thanks!

@sander76
Copy link
Author

@sander76 sander76 commented Dec 7, 2016

Greatn

@sander76
Copy link
Author

@sander76 sander76 commented Dec 7, 2016

Great!

jsonn pushed a commit to jsonn/pkgsrc that referenced this issue Jan 15, 2017
Version 0.34
------------

Released on 2016-12-21.

Bug fixes:

* `#398 <https://github.com/Kozea/WeasyPrint/issues/398>`_:
  Honor the presentational_hints option for PDFs.
* `#399 <https://github.com/Kozea/WeasyPrint/pull/399>`_:
  Avoid CairoSVG-2.0.0rc* on Python 2.
* `#396 <https://github.com/Kozea/WeasyPrint/issues/396>`_:
  Correctly close files open by mkstemp.
* `#403 <https://github.com/Kozea/WeasyPrint/issues/403>`_:
  Cast the number of columns into int.
* Fix multi-page multi-columns and add related tests.


Version 0.33
------------

Released on 2016-11-28.

New features:

* `#393 <https://github.com/Kozea/WeasyPrint/issues/393>`_:
  Add tests on MacOS.
* `#370 <https://github.com/Kozea/WeasyPrint/issues/370>`_:
  Enable @font-face on MacOS.

Bug fixes:

* `#389 <https://github.com/Kozea/WeasyPrint/issues/389>`_:
  Always update resume_at when splitting lines.
* `#394 <https://github.com/Kozea/WeasyPrint/issues/394>`_:
  Don't build universal wheels.
* `#388 <https://github.com/Kozea/WeasyPrint/issues/388>`_:
  Fix logic when finishing block formatting context.


Version 0.32
------------

Released on 2016-11-17.

New features:

* `#28 <https://github.com/Kozea/WeasyPrint/issues/28>`_:
  Support @font-face on Linux.
* Support CSS fonts level 3 almost entirely, including OpenType features.
* `#253 <https://github.com/Kozea/WeasyPrint/issues/253>`_:
  Support presentational hints (optional).
* Support break-after, break-before and break-inside for pages and columns.
* `#384 <https://github.com/Kozea/WeasyPrint/issues/384>`_:
  Major performance boost.

Bux fixes:

* `#368 <https://github.com/Kozea/WeasyPrint/issues/368>`_:
  Respect white-space for shrink-to-fit.
* `#382 <https://github.com/Kozea/WeasyPrint/issues/382>`_:
  Fix the preferred width for column groups.
* Handle relative boxes in column-layout boxes.

Documentation:

* Add more and more documentation about Windows installation.
* `#355 <https://github.com/Kozea/WeasyPrint/issues/355>`_:
  Add fonts requirements for tests.


Version 0.31
------------

Released on 2016-08-28.

New features:

* `#124 <https://github.com/Kozea/WeasyPrint/issues/124>`_:
  Add MIME sniffing for images.
* `#60 <https://github.com/Kozea/WeasyPrint/issues/60>`_:
  CSS Multi-column Layout.
* `#197 <https://github.com/Kozea/WeasyPrint/pull/197>`_:
  Add hyphens at line breaks activated by a soft hyphen.

Bux fixes:

* `#132 <https://github.com/Kozea/WeasyPrint/pull/132>`_:
  Fix Python 3 compatibility on Windows.

Documentation:

* `#329 <https://github.com/Kozea/WeasyPrint/issues/329>`_:
  Add documentation about installation on Windows.


Version 0.30
------------

Released on 2016-07-18.

WeasyPrint now depends on html5lib-0.999999999.

Bux fixes:

* Fix Acid2
* `#325 <https://github.com/Kozea/WeasyPrint/issues/325>`_:
  Cutting lines is broken in page margin boxes.
* `#334 <https://github.com/Kozea/WeasyPrint/issues/334>`_:
  Newest html5lib 0.999999999 breaks rendering.


Version 0.29
------------

Released on 2016-06-17.

Bug fixes:

* `#263 <https://github.com/Kozea/WeasyPrint/pull/263>`_:
  Don't crash with floats with percents in positions.
* `#323 <https://github.com/Kozea/WeasyPrint/pull/323>`_:
  Fix CairoSVG 2.0 pre-release dependency in Python 2.x.


Version 0.28
------------

Released on 2016-05-16.

Bug fixes:

* `#189 <https://github.com/Kozea/WeasyPrint/issues/189>`_:
  ``white-space: nowrap`` still wraps on hyphens
* `#305 <https://github.com/Kozea/WeasyPrint/issues/305>`_:
  Fix crashes on some tables
* Don't crash when transform matrix isn't invertible
* Don't crash when rendering ratio-only SVG images
* Fix margins and borders on some tables


Version 0.27
------------

Released on 2016-04-08.

New features:

* `#295 <https://github.com/Kozea/WeasyPrint/pull/295>`_:
  Support the 'rem' unit.
* `#299 <https://github.com/Kozea/WeasyPrint/pull/299>`_:
  Enhance the support of SVG images.

Bug fixes:

* `#307 <https://github.com/Kozea/WeasyPrint/issues/307>`_:
  Fix the layout of cells larger than their tables.

Documentation:

* The website is now on GitHub Pages, the documentation is on Read the Docs.
* `#297 <https://github.com/Kozea/WeasyPrint/issues/297>`_:
  Rewrite the CSS chapter of the documentation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.