Need to sanitize comments

[pussinboot]

UnicodeEncodeError: 'charmap' codec can't encode character '\u200a' in position 14: character maps to <undefined>
windows 7, using cmd and python 3.5

while browsing comments, when reaching an unsupported character the entire screen fills up with a trace of the error

[donnemartin]

Good catch. Curious if you remember the post that displays this issue? Would be easier for me to debug :)

I've tried a number of posts but have not run across this--I'll keep an eye out.

[pussinboot]

Hi, I couldn't find the exact post but I did manage to replicate the error

to replicate follow these steps

hn user JasonNils

view 1 -c

(i posted a test comment on the "Browse Hacker News Like a Haxor" submission )

and here's the full error
Traceback (most recent call last): File "c:\users\[username]\appdata\local\continuum\anaconda3\lib\runpy.py", line 170, in _run_module_as_main "__main__", mod_spec) File "c:\users\[username]\appdata\local\continuum\anaconda3\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\Users\[username]\AppData\Local\Continuum\Anaconda3\Scripts\hn.exe\__main__.py", line 9, in <module> File "c:\users\[username]\appdata\local\continuum\anaconda3\lib\site-packages\haxor_news\main_cli.py", line 25, in cli haxor_news.cli() File "c:\users\[username]\appdata\local\continuum\anaconda3\lib\site-packages\click\core.py", line 716, in __call__ return self.main(*args, **kwargs) File "c:\users\[username]\appdata\local\continuum\anaconda3\lib\site-packages\click\core.py", line 696, in main rv = self.invoke(ctx) File "c:\users\[username]\appdata\local\continuum\anaconda3\lib\site-packages\click\core.py", line 1060, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "c:\users\[username]\appdata\local\continuum\anaconda3\lib\site-packages\click\core.py", line 889, in invoke return ctx.invoke(self.callback, **ctx.params) File "c:\users\[username]\appdata\local\continuum\anaconda3\lib\site-packages\click\core.py", line 534, in invoke return callback(*args, **kwargs) File "c:\users\[username]\appdata\local\continuum\anaconda3\lib\site-packages\click\decorators.py", line 64, in new_func return ctx.invoke(f, obj, *args[1:], **kwargs) File "c:\users\[username]\appdata\local\continuum\anaconda3\lib\site-packages\click\core.py", line 534, in invoke return callback(*args, **kwargs) File "c:\users\[username]\appdata\local\continuum\anaconda3\lib\site-packages\haxor_news\hacker_news_cli.py", line 367, in view browser) File "c:\users\[username]\appdata\local\continuum\anaconda3\lib\site-packages\haxor_news\hacker_news.py", line 634, in view_setup browser) File "c:\users\[username]\appdata\local\continuum\anaconda3\lib\site-packages\haxor_news\hacker_news.py", line 552, in view comments_hide_non_matching=comments_hide_non_matching) File "c:\users\[username]\appdata\local\continuum\anaconda3\lib\site-packages\haxor_news\hacker_news.py", line 255, in print_comments depth=depth) File "c:\users\[username]\appdata\local\continuum\anaconda3\lib\site-packages\haxor_news\hacker_news.py", line 243, in print_comments self.print_comment(item, regex_query, comments_hide_non_matching, depth) File "c:\users\[username]\appdata\local\continuum\anaconda3\lib\site-packages\haxor_news\hacker_news.py", line 216, in print_comment click.echo(formatted_comment, color=True) File "c:\users\[username]\appdata\local\continuum\anaconda3\lib\site-packages\click\utils.py", line 259, in echo file.write(message) File "c:\users\[username]\appdata\local\continuum\anaconda3\lib\site-packages\click\_compat.py", line 571, in _safe_write return _write(s) File "c:\users\[username]\appdata\local\continuum\anaconda3\lib\site-packages\colorama\ansitowin32.py", line 36, in write self.__convertor.write(text) File "c:\users\[username]\appdata\local\continuum\anaconda3\lib\site-packages\colorama\ansitowin32.py", line 133, in write self.write_and_convert(text) File "c:\users\[username]\appdata\local\continuum\anaconda3\lib\site-packages\colorama\ansitowin32.py", line 161, in write_and_convert self.write_plain_text(text, cursor, len(text)) File "c:\users\[username]\appdata\local\continuum\anaconda3\lib\site-packages\colorama\ansitowin32.py", line 166, in write_plain_text self.wrapped.write(text[start:end]) File "c:\users\[username]\appdata\local\continuum\anaconda3\lib\encodings\cp1252.py", line 19, in encode return codecs.charmap_encode(input,self.errors,encoding_table)[0]UnicodeEncodeError: 'charmap' codec can't encode character '\u200a' in position10: character maps to <undefined>

[donnemartin]

Fixed in 3e72365.

I plan to push out a release to PyPI soon.