Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Python 3 compatibility on Windows #132

Closed
wants to merge 1 commit into from
Closed

Conversation

@SAPikachu
Copy link
Contributor

SAPikachu commented Oct 31, 2013

In Python 3, pathname2url accepts str (which is aliased as unicode in compat.py), so path should not be encoded.

In Python 3, pathname2url accepts str (which is aliased as unicode in compat.py), so path should not be encoded.
@SimonSapin
Copy link
Member

SimonSapin commented Oct 31, 2013

I’ll have to review this more in depth, and possibly make larger changes. Filename handling across platform and Python version is surprisingly tricky.

Can you give a specific example where this was a problem? Please include as many details about the environment (OS, locale, filesystem encoding, Python version) and data (Python code, HTML, CSS) as you can.

@SAPikachu
Copy link
Contributor Author

SAPikachu commented Oct 31, 2013

Actually 0.19.2 crashes on my system even without any arguments, here is the crash output:

C:\Users\SAPikachu\Dropbox\work\mlslink\expire_search>py -3 -mweasyprint
Traceback (most recent call last):
  File "C:\Python33\lib\runpy.py", line 140, in _run_module_as_main
    mod_name, loader, code, fname = _get_module_details(mod_name)
  File "C:\Python33\lib\runpy.py", line 110, in _get_module_details
    return _get_module_details(pkg_main_name)
  File "C:\Python33\lib\runpy.py", line 102, in _get_module_details
    loader = get_loader(mod_name)
  File "C:\Python33\lib\pkgutil.py", line 482, in get_loader
    return find_loader(fullname)
  File "C:\Python33\lib\pkgutil.py", line 499, in find_loader
    pkg = importlib.import_module(pkg_name)
  File "C:\Python33\lib\importlib\__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1586, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1567, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1534, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 586, in _check_name_wrapper
  File "<frozen importlib._bootstrap>", line 1024, in load_module
  File "<frozen importlib._bootstrap>", line 1005, in load_module
  File "<frozen importlib._bootstrap>", line 562, in module_for_loader_wrapper
  File "<frozen importlib._bootstrap>", line 870, in _load_module
  File "<frozen importlib._bootstrap>", line 313, in _call_with_frames_removed
  File "C:\Python33\lib\site-packages\weasyprint-0.19.2-py3.3.egg\weasyprint\__init__.py", line 305, in <module>
    from .html import find_base_url, HTML5_UA_STYLESHEET
  File "C:\Python33\lib\site-packages\weasyprint-0.19.2-py3.3.egg\weasyprint\html.py", line 34, in <module>
    filename=os.path.join(os.path.dirname(__file__), 'css', 'html5_ua.css'))
  File "C:\Python33\lib\site-packages\weasyprint-0.19.2-py3.3.egg\weasyprint\__init__.py", line 219, in __init__
    check_css_mime_type=_check_mime_type,)
  File "C:\Python33\lib\site-packages\weasyprint-0.19.2-py3.3.egg\weasyprint\__init__.py", line 268, in _select_source
    base_url = path2url(filename)
  File "C:\Python33\lib\site-packages\weasyprint-0.19.2-py3.3.egg\weasyprint\urls.py", line 81, in path2url
    path = pathname2url(path)
  File "C:\Python33\lib\nturl2path.py", line 46, in pathname2url
    if not ':' in p:
TypeError: Type str doesn't support the buffer API

C:\Users\SAPikachu\Dropbox\work\mlslink\expire_search>py -3 --version
Python 3.3.2

Information about my system:

  • OS: Windows 8.1 Pro
  • Python version: 3.3.2
  • Language: English
  • Locale: Chinese
  • System codepage: 936 (GBK, Chinese Simplified)
  • Filesystem encoding: AFAIK python 3 uses unicode version of Windows API so this should be UTF-16, but I think this is irrelevant in this case because str in python 3 is always in unicode, and python will handle encoding conversion internally.
@rhvonlehe
Copy link

rhvonlehe commented Jun 28, 2016

@liZe
Copy link
Member

liZe commented Jul 4, 2016

I'm trying to make WP work on Windows with Python 2.x and 3.x (ie. fixing #331 #330 #132 #329).

As said by @rhvonlehe, the Windows version of pathname2url requires unicode and not bytes (it's not really documented and works on Linux, well…). But I'd really prefer to keep returning bytes in path2url, as it's probably safer.

So, instead of testing the Python version, I'd prefer to call path = pathname2url(path) before path = path.encode(FILESYSTEM_ENCODING). @SimonSapin what do you think of that?

@SimonSapin
Copy link
Member

SimonSapin commented Jul 4, 2016

So… this probably requires more time to research than I have right now. But I vaguely remember thinking that we’d be better of implementing the file: URL scheme ourselves rather than leave it to urllib. Stay in Unicode all the way, and give a Unicode string to open().

@liZe
Copy link
Member

liZe commented Jul 5, 2016

Stay in Unicode all the way, and give a Unicode string to open().

@SimonSapin It's better for Python 3, but we probably need to keep bytes for Python 2 (I suppose that the path.encode had been added for a good reason). So: we merge the PR as is?

I'm trying to make WP work on Windows with Python 2.x and 3.x

@rhvonlehe @ChameleonRed I've installed WeasyPrint on Windows 10 64 bit following these steps:

I've met the problem in #330 during my different tests. What seems to be important is:

  • To compile some packages (which ones?) on your version of Windows. That's what pip does, and that's why it relies on Visual C++.
  • To use 64 bit packages everywhere.

Following these two rules on other versions of Windows (including 32 bit with 32 bit packages) should be OK. With this PR, it should even work on Windows with Python 3.5.

If it works for @rhvonlehe and @ChameleonRed, I can change the documentation.

@SimonSapin
Copy link
Member

SimonSapin commented Jul 5, 2016

(I suppose that the path.encode had been added for a good reason)

Haha, that’s a strong supposition. Python 2 supports Unicode in open() fine, doesn’t it?

@SimonSapin
Copy link
Member

SimonSapin commented Jul 5, 2016

To clarify, my suggestion to stay in Unicode all the way is if we change default_url_fetcher to have a special case for file: URLs (like there’s one for data:) where we handle them ourselves instead of giving them to urlopen(): split to extract the path component, reject if host is not empty or localhost, percent-decode, then call open().

@ghedin
Copy link

ghedin commented Jul 7, 2016

Hello folks,

First of all I would like to congratulate @SimonSapin and all the project's collaborators. This is a great piece of software 👍

I also was unable to use the library with Python3 on Windows 7. After applying the pull request suggested by @SAPikachu it just works. I have tested it with a few simple HTML files and could successfully convert them to PDF.

I'm not with much spare time at the moment, but I'd like to contribute by applying @SimonSapin suggestion (stay Unicode all the way). Since I did not look at the code yet, I can't imagine how big that change would be, but I will take a look at it asap.

@rhvonlehe
Copy link

rhvonlehe commented Jul 7, 2016

I understand you'd like me to install using 64-bit versions of everything
to see if that helps with Python 3.5 compatibility on Windows. I can do
that, but it might be this weekend before I get to it.

EDIT: @liZe FWIW, I also have had success using weasyprint using Python 2.7. I am doubtful that simply installing 64-bit versions of Python 3.5 and GTK-3 will have an effect on #132, but I will try on my home PC this weekend hopefully.

On Thu, Jul 7, 2016 at 7:41 AM, ghedin notifications@github.com wrote:

Hello folks,

First of all I would like to congratulate @SimonSapin
https://github.com/SimonSapin and all the project's collaborators. This
is a great piece of software 👍

I also was unable to use the library with Python3 on Windows 7. After
applying the pull request suggested by @SAPikachu
https://github.com/SAPikachu it just works. I have tested it with a few
simple HTML files and could successfully convert them to PDF.

I'm not with much spare time at the moment, but I'd like to contribute by
applying @SimonSapin https://github.com/SimonSapin suggestion (stay
Unicode all the way). Since I did not look at the code yet, I can't imagine
how big that change would be, but I will take a look at it asap.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#132 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AF4Nd8pFp9VTbjqbRVm4l9wZxzKj8jkgks5qTPP8gaJpZM4BJ69n
.

@liZe
Copy link
Member

liZe commented Jul 7, 2016

I understand you'd like me to install using 64-bit versions of everything
to see if that helps with Python 3.5 compatibility on Windows.

:) not exactly! Here's what I propose:

  1. you can try follow the installation guide I provided to fix #329 and #330,
  2. I write the new version of this PR to fix #132 and #331 / #332 / #261.

With these two points solved, we can consider the Windows support as quite good!

@rhvonlehe
Copy link

rhvonlehe commented Jul 7, 2016

@liZe Ah, I see. I think I can safely say I don't experience issue #329, using either 32-bit Python 2.7 or 32-bit Python 3.5, and 32-bit GTK (on 64-bit Windows 10). I think your installation guide clarifications will definitely help, though. I had the best luck following instructions here [1], rather than the current weasyprint instructions. I am willing to follow your instructions explicitly on my home machine (current no weasy installed), though, if that helps. Looking forward to your new PR for #132, thanks!

[1] https://gist.github.com/doobeh/3188318

@liZe liZe changed the title Fix Python 3 compatibility Fix Python 3 compatibility on Windows Jul 28, 2016
This was referenced Aug 12, 2016
@dom96
Copy link

dom96 commented Aug 15, 2016

Don't have much to contribute to the discussion except that I ran into the same problem and applying this PR fixed it for me. (Windows 7 x86_64, Python 3.4 (Installed via Anaconda), let me know if you need more info).

Hope this helps!

@liZe liZe mentioned this pull request Aug 16, 2016
@liZe
Copy link
Member

liZe commented Aug 16, 2016

@SAPikachu @rhvonlehe @dom96 I've written and tested #345. Could you please check that it works for you?

@liZe
Copy link
Member

liZe commented Aug 17, 2016

Fixed now that #345 is fixed. Thanks for your help!

@liZe liZe closed this Aug 17, 2016
@liZe liZe added this to the v0.31 milestone Aug 26, 2016
jsonn pushed a commit to jsonn/pkgsrc that referenced this pull request Jan 15, 2017
Version 0.34
------------

Released on 2016-12-21.

Bug fixes:

* `#398 <https://github.com/Kozea/WeasyPrint/issues/398>`_:
  Honor the presentational_hints option for PDFs.
* `#399 <https://github.com/Kozea/WeasyPrint/pull/399>`_:
  Avoid CairoSVG-2.0.0rc* on Python 2.
* `#396 <https://github.com/Kozea/WeasyPrint/issues/396>`_:
  Correctly close files open by mkstemp.
* `#403 <https://github.com/Kozea/WeasyPrint/issues/403>`_:
  Cast the number of columns into int.
* Fix multi-page multi-columns and add related tests.


Version 0.33
------------

Released on 2016-11-28.

New features:

* `#393 <https://github.com/Kozea/WeasyPrint/issues/393>`_:
  Add tests on MacOS.
* `#370 <https://github.com/Kozea/WeasyPrint/issues/370>`_:
  Enable @font-face on MacOS.

Bug fixes:

* `#389 <https://github.com/Kozea/WeasyPrint/issues/389>`_:
  Always update resume_at when splitting lines.
* `#394 <https://github.com/Kozea/WeasyPrint/issues/394>`_:
  Don't build universal wheels.
* `#388 <https://github.com/Kozea/WeasyPrint/issues/388>`_:
  Fix logic when finishing block formatting context.


Version 0.32
------------

Released on 2016-11-17.

New features:

* `#28 <https://github.com/Kozea/WeasyPrint/issues/28>`_:
  Support @font-face on Linux.
* Support CSS fonts level 3 almost entirely, including OpenType features.
* `#253 <https://github.com/Kozea/WeasyPrint/issues/253>`_:
  Support presentational hints (optional).
* Support break-after, break-before and break-inside for pages and columns.
* `#384 <https://github.com/Kozea/WeasyPrint/issues/384>`_:
  Major performance boost.

Bux fixes:

* `#368 <https://github.com/Kozea/WeasyPrint/issues/368>`_:
  Respect white-space for shrink-to-fit.
* `#382 <https://github.com/Kozea/WeasyPrint/issues/382>`_:
  Fix the preferred width for column groups.
* Handle relative boxes in column-layout boxes.

Documentation:

* Add more and more documentation about Windows installation.
* `#355 <https://github.com/Kozea/WeasyPrint/issues/355>`_:
  Add fonts requirements for tests.


Version 0.31
------------

Released on 2016-08-28.

New features:

* `#124 <https://github.com/Kozea/WeasyPrint/issues/124>`_:
  Add MIME sniffing for images.
* `#60 <https://github.com/Kozea/WeasyPrint/issues/60>`_:
  CSS Multi-column Layout.
* `#197 <https://github.com/Kozea/WeasyPrint/pull/197>`_:
  Add hyphens at line breaks activated by a soft hyphen.

Bux fixes:

* `#132 <https://github.com/Kozea/WeasyPrint/pull/132>`_:
  Fix Python 3 compatibility on Windows.

Documentation:

* `#329 <https://github.com/Kozea/WeasyPrint/issues/329>`_:
  Add documentation about installation on Windows.


Version 0.30
------------

Released on 2016-07-18.

WeasyPrint now depends on html5lib-0.999999999.

Bux fixes:

* Fix Acid2
* `#325 <https://github.com/Kozea/WeasyPrint/issues/325>`_:
  Cutting lines is broken in page margin boxes.
* `#334 <https://github.com/Kozea/WeasyPrint/issues/334>`_:
  Newest html5lib 0.999999999 breaks rendering.


Version 0.29
------------

Released on 2016-06-17.

Bug fixes:

* `#263 <https://github.com/Kozea/WeasyPrint/pull/263>`_:
  Don't crash with floats with percents in positions.
* `#323 <https://github.com/Kozea/WeasyPrint/pull/323>`_:
  Fix CairoSVG 2.0 pre-release dependency in Python 2.x.


Version 0.28
------------

Released on 2016-05-16.

Bug fixes:

* `#189 <https://github.com/Kozea/WeasyPrint/issues/189>`_:
  ``white-space: nowrap`` still wraps on hyphens
* `#305 <https://github.com/Kozea/WeasyPrint/issues/305>`_:
  Fix crashes on some tables
* Don't crash when transform matrix isn't invertible
* Don't crash when rendering ratio-only SVG images
* Fix margins and borders on some tables


Version 0.27
------------

Released on 2016-04-08.

New features:

* `#295 <https://github.com/Kozea/WeasyPrint/pull/295>`_:
  Support the 'rem' unit.
* `#299 <https://github.com/Kozea/WeasyPrint/pull/299>`_:
  Enhance the support of SVG images.

Bug fixes:

* `#307 <https://github.com/Kozea/WeasyPrint/issues/307>`_:
  Fix the layout of cells larger than their tables.

Documentation:

* The website is now on GitHub Pages, the documentation is on Read the Docs.
* `#297 <https://github.com/Kozea/WeasyPrint/issues/297>`_:
  Rewrite the CSS chapter of the documentation.
@TheEasyProgrammer
Copy link

TheEasyProgrammer commented Apr 11, 2017

I am getting cool to work in ubuntu box but I am struggling in this issue in windows box. I have installed gtk and have path in environment variables and also installed the visual c++ compiler for python. But no luck. Please help me on this @SAPikachu @liZe
libcairo.so.2 os error 0x7e in python

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

7 participants
You can’t perform that action at this time.