Fix Python 3 compatibility on Windows

[SAPikachu]

In Python 3, pathname2url accepts str (which is aliased as unicode in compat.py), so path should not be encoded.

[SimonSapin]

I’ll have to review this more in depth, and possibly make larger changes. Filename handling across platform and Python version is surprisingly tricky.

Can you give a specific example where this was a problem? Please include as many details about the environment (OS, locale, filesystem encoding, Python version) and data (Python code, HTML, CSS) as you can.

[SAPikachu]

Actually 0.19.2 crashes on my system even without any arguments, here is the crash output:

C:\Users\SAPikachu\Dropbox\work\mlslink\expire_search>py -3 -mweasyprint
Traceback (most recent call last):
  File "C:\Python33\lib\runpy.py", line 140, in _run_module_as_main
    mod_name, loader, code, fname = _get_module_details(mod_name)
  File "C:\Python33\lib\runpy.py", line 110, in _get_module_details
    return _get_module_details(pkg_main_name)
  File "C:\Python33\lib\runpy.py", line 102, in _get_module_details
    loader = get_loader(mod_name)
  File "C:\Python33\lib\pkgutil.py", line 482, in get_loader
    return find_loader(fullname)
  File "C:\Python33\lib\pkgutil.py", line 499, in find_loader
    pkg = importlib.import_module(pkg_name)
  File "C:\Python33\lib\importlib\__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1586, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1567, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1534, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 586, in _check_name_wrapper
  File "<frozen importlib._bootstrap>", line 1024, in load_module
  File "<frozen importlib._bootstrap>", line 1005, in load_module
  File "<frozen importlib._bootstrap>", line 562, in module_for_loader_wrapper
  File "<frozen importlib._bootstrap>", line 870, in _load_module
  File "<frozen importlib._bootstrap>", line 313, in _call_with_frames_removed
  File "C:\Python33\lib\site-packages\weasyprint-0.19.2-py3.3.egg\weasyprint\__init__.py", line 305, in <module>
    from .html import find_base_url, HTML5_UA_STYLESHEET
  File "C:\Python33\lib\site-packages\weasyprint-0.19.2-py3.3.egg\weasyprint\html.py", line 34, in <module>
    filename=os.path.join(os.path.dirname(__file__), 'css', 'html5_ua.css'))
  File "C:\Python33\lib\site-packages\weasyprint-0.19.2-py3.3.egg\weasyprint\__init__.py", line 219, in __init__
    check_css_mime_type=_check_mime_type,)
  File "C:\Python33\lib\site-packages\weasyprint-0.19.2-py3.3.egg\weasyprint\__init__.py", line 268, in _select_source
    base_url = path2url(filename)
  File "C:\Python33\lib\site-packages\weasyprint-0.19.2-py3.3.egg\weasyprint\urls.py", line 81, in path2url
    path = pathname2url(path)
  File "C:\Python33\lib\nturl2path.py", line 46, in pathname2url
    if not ':' in p:
TypeError: Type str doesn't support the buffer API

C:\Users\SAPikachu\Dropbox\work\mlslink\expire_search>py -3 --version
Python 3.3.2

Information about my system:

• OS: Windows 8.1 Pro
• Python version: 3.3.2
• Language: English
• Locale: Chinese
• System codepage: 936 (GBK, Chinese Simplified)
• Filesystem encoding: AFAIK python 3 uses unicode version of Windows API so this should be UTF-16, but I think this is irrelevant in this case because str in python 3 is always in unicode, and python will handle encoding conversion internally.

[rhvonlehe]

I'd like to see this compatibility cleaned up as well in a future release. See stackoverflow issues [1] and [2]:
[1] http://stackoverflow.com/questions/37994733/weasyprint-usage-with-python-3-x-on-windows/38082024#38082024
[2] http://stackoverflow.com/questions/38081511/need-help-understanding-bytes-like-object-vs-str-in-python-3-5

[liZe]

I'm trying to make WP work on Windows with Python 2.x and 3.x (ie. fixing #331 #330 #132 #329).

As said by @rhvonlehe, the Windows version of pathname2url requires unicode and not bytes (it's not really documented and works on Linux, well…). But I'd really prefer to keep returning bytes in path2url, as it's probably safer.

So, instead of testing the Python version, I'd prefer to call path = pathname2url(path) before path = path.encode(FILESYSTEM_ENCODING). @SimonSapin what do you think of that?

[SimonSapin]

So… this probably requires more time to research than I have right now. But I vaguely remember thinking that we’d be better of implementing the file: URL scheme ourselves rather than leave it to urllib. Stay in Unicode all the way, and give a Unicode string to open().

[liZe]

Stay in Unicode all the way, and give a Unicode string to open().

@SimonSapin It's better for Python 3, but we probably need to keep bytes for Python 2 (I suppose that the path.encode had been added for a good reason). So: we merge the PR as is?

I'm trying to make WP work on Windows with Python 2.x and 3.x

@rhvonlehe @ChameleonRed I've installed WeasyPrint on Windows 10 64 bit following these steps:

• Install Python 2.7.x 64 bit with python in PATH.
• Install GTK-3 64 bit.
• Reboot (true story).
• Install Visual C++ compiler for Python 2.7.
• Install WeasyPrint with python -m pip install weasyprint.
• Test with python -m weasyprint http://weasyprint.org weasyprint.pdf.

I've met the problem in #330 during my different tests. What seems to be important is:

• To compile some packages (which ones?) on your version of Windows. That's what pip does, and that's why it relies on Visual C++.
• To use 64 bit packages everywhere.

Following these two rules on other versions of Windows (including 32 bit with 32 bit packages) should be OK. With this PR, it should even work on Windows with Python 3.5.

If it works for @rhvonlehe and @ChameleonRed, I can change the documentation.

[SimonSapin]

(I suppose that the path.encode had been added for a good reason)

Haha, that’s a strong supposition. Python 2 supports Unicode in open() fine, doesn’t it?

[SimonSapin]

To clarify, my suggestion to stay in Unicode all the way is if we change default_url_fetcher to have a special case for file: URLs (like there’s one for data:) where we handle them ourselves instead of giving them to urlopen(): split to extract the path component, reject if host is not empty or localhost, percent-decode, then call open().

[ghedin]

Hello folks,

First of all I would like to congratulate @SimonSapin and all the project's collaborators. This is a great piece of software 👍

I also was unable to use the library with Python3 on Windows 7. After applying the pull request suggested by @SAPikachu it just works. I have tested it with a few simple HTML files and could successfully convert them to PDF.

I'm not with much spare time at the moment, but I'd like to contribute by applying @SimonSapin suggestion (stay Unicode all the way). Since I did not look at the code yet, I can't imagine how big that change would be, but I will take a look at it asap.

[rhvonlehe]

I understand you'd like me to install using 64-bit versions of everything
to see if that helps with Python 3.5 compatibility on Windows. I can do
that, but it might be this weekend before I get to it.

EDIT: @liZe FWIW, I also have had success using weasyprint using Python 2.7. I am doubtful that simply installing 64-bit versions of Python 3.5 and GTK-3 will have an effect on #132, but I will try on my home PC this weekend hopefully.

On Thu, Jul 7, 2016 at 7:41 AM, ghedin notifications@github.com wrote:

Hello folks,

First of all I would like to congratulate @SimonSapin
https://github.com/SimonSapin and all the project's collaborators. This
is a great piece of software 👍

I also was unable to use the library with Python3 on Windows 7. After
applying the pull request suggested by @SAPikachu
https://github.com/SAPikachu it just works. I have tested it with a few
simple HTML files and could successfully convert them to PDF.

I'm not with much spare time at the moment, but I'd like to contribute by
applying @SimonSapin https://github.com/SimonSapin suggestion (stay
Unicode all the way). Since I did not look at the code yet, I can't imagine
how big that change would be, but I will take a look at it asap.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#132 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AF4Nd8pFp9VTbjqbRVm4l9wZxzKj8jkgks5qTPP8gaJpZM4BJ69n
.

[liZe]

I understand you'd like me to install using 64-bit versions of everything
to see if that helps with Python 3.5 compatibility on Windows.

:) not exactly! Here's what I propose:

1. you can try follow the installation guide I provided to fix I could not install windows version of weasyprint because lack of information ... #329 and Weasyprint fails on Windows 7/Python 2.7.x #330,
2. I write the new version of this PR to fix Fix Python 3 compatibility on Windows #132 and a bytes-like object is required, not 'str' while importing HTML from weasyprint #331 / a bytes-like object is required, not 'str' while importing HTML from weasyprint #332 / Under Windows pathname2url.pathname2url raises "TypeError: 'str' does not support the buffer interface" #261.

With these two points solved, we can consider the Windows support as quite good!

[rhvonlehe]

@liZe Ah, I see. I think I can safely say I don't experience issue #329, using either 32-bit Python 2.7 or 32-bit Python 3.5, and 32-bit GTK (on 64-bit Windows 10). I think your installation guide clarifications will definitely help, though. I had the best luck following instructions here [1], rather than the current weasyprint instructions. I am willing to follow your instructions explicitly on my home machine (current no weasy installed), though, if that helps. Looking forward to your new PR for #132, thanks!

[1] https://gist.github.com/doobeh/3188318

[dom96]

Don't have much to contribute to the discussion except that I ran into the same problem and applying this PR fixed it for me. (Windows 7 x86_64, Python 3.4 (Installed via Anaconda), let me know if you need more info).

Hope this helps!

[liZe]

@SAPikachu @rhvonlehe @dom96 I've written and tested #345. Could you please check that it works for you?

[liZe]

Fixed now that #345 is fixed. Thanks for your help!

[TheEasyProgrammer]

I am getting cool to work in ubuntu box but I am struggling in this issue in windows box. I have installed gtk and have path in environment variables and also installed the visual c++ compiler for python. But no luck. Please help me on this @SAPikachu @liZe
libcairo.so.2 os error 0x7e in python