Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix error with non ascii #11

Closed
sserrano44 opened this issue Oct 25, 2011 · 3 comments
Closed

fix error with non ascii #11

sserrano44 opened this issue Oct 25, 2011 · 3 comments

Comments

@sserrano44
Copy link

This patch worked for me

--- a/pynliner/__init__.py
+++ b/pynliner/__init__.py
@@ -208,7 +208,7 @@ class Pynliner(object):

         Returns self.output
         """
-        self.output = unicode(str(self.soup))
+        self.output = unicode(self.soup.renderContents(), 'utf-8')
         return self.output

def fromURL(url, log=None):
@rennat
Copy link
Owner

rennat commented Nov 28, 2011

This is a minor change but please fork and submit a pull request so that you are properly attributed for your commits :)

@peteroconnor
Copy link

While the above fix did get rid of an error at that particular location, I got another error when later processing the html.

(The New Error)

File "/home/www-data/web2py/gluon/tools.py", line 363, in send
  html = html.decode(encoding).encode('utf-8')
File "/usr/lib/python2.6/encodings/utf_8.py", line 16, in decode
  return codecs.utf_8_decode(input, errors, True)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u7528' in position 293: ordinal not in range(128)

I used the following patch and fixed both errors. I am not an expert in unicode. Just try and error.

211c211
<         self.output = str(self.soup.renderContents())
---
>         self.output = unicode(str(self.soup))

@zz85 zz85 mentioned this issue Oct 8, 2012
@rennat
Copy link
Owner

rennat commented Apr 26, 2013

This was related to some odd behavior in the current (at the time) version of BeautifulSoup and newer versions allow us to go directly to unicode.

(the BeautifulSoup version requirements for pynliner have been modified to reflect this in tag 0.5.0)

@rennat rennat closed this as completed Apr 26, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants