Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for converting notebooks that contain unicode characters. #3870

Merged
merged 1 commit into from
Aug 2, 2013

Conversation

astrofrog
Copy link
Contributor

I tried converting a notebook that had a ü in it using nbconvert, and get:

$ ipython3 nbconvert *.ipynb --to latex --post PDF
[NbConvertApp] Using existing profile dir: '/Users/tom/.ipython/profile_default'
[NbConvertApp] Converting notebook 00. About the course.ipynb to latex
[NbConvertApp] Support files will be in 00. About the course_files/
[NbConvertApp] Loaded template latex_article.tplx
[NbConvertApp] Writing 17249 bytes to ./00. About the course.tex
b'stitut f\xc3\xbcr\nAstronomi'
Traceback (most recent call last):
  File "/Users/tom/Library/Python/3.3/bin/ipython3", line 9, in <module>
    load_entry_point('ipython==1.0.0-dev', 'console_scripts', 'ipython3')()
  File "/Users/tom/Library/Python/3.3/lib/python/site-packages/ipython-1.0.0_dev-py3.3.egg/IPython/__init__.py", line 118, in start_ipython
    return launch_new_instance(argv=argv, **kwargs)
  File "/Users/tom/Library/Python/3.3/lib/python/site-packages/ipython-1.0.0_dev-py3.3.egg/IPython/config/application.py", line 539, in launch_instance
    app.start()
  File "/Users/tom/Library/Python/3.3/lib/python/site-packages/ipython-1.0.0_dev-py3.3.egg/IPython/terminal/ipapp.py", line 355, in start
    return self.subapp.start()
  File "/Users/tom/Library/Python/3.3/lib/python/site-packages/ipython-1.0.0_dev-py3.3.egg/IPython/nbconvert/nbconvertapp.py", line 266, in start
    self.convert_notebooks()
  File "/Users/tom/Library/Python/3.3/lib/python/site-packages/ipython-1.0.0_dev-py3.3.egg/IPython/nbconvert/nbconvertapp.py", line 305, in convert_notebooks
    write_resultes = self.writer.write(output, resources, notebook_name=notebook_name)
  File "/Users/tom/Library/Python/3.3/lib/python/site-packages/ipython-1.0.0_dev-py3.3.egg/IPython/nbconvert/writers/files.py", line 109, in write
    f.write(output)
UnicodeEncodeError: 'ascii' codec can't encode character '\xfc' in position 11338: ordinal not in range(128)

The attached code seems to fix this, but understanding the intricacies of unicode isn't my strongest point, so let me know if it's the completely wrong approach :)

@takluyver
Copy link
Member

We'll need to check whether output is always unicode, or if there's some way that it can be a byte string.

Also, it would be slightly neater to use with io.open(... encoding='utf-8') instead of the explicit .encode() call.

@astrofrog
Copy link
Contributor Author

@takluyver - thanks, I'll give those suggestions a try

@minrk
Copy link
Member

minrk commented Aug 1, 2013

output should always be unicode

@minrk
Copy link
Member

minrk commented Aug 1, 2013

Go ahead make make the io.open(...encoding='utf-8') change, and this should be good to go.

@minrk
Copy link
Member

minrk commented Aug 2, 2013

@astrofrog want to make that change?

minrk added a commit that referenced this pull request Aug 2, 2013
Fix for converting notebooks that contain unicode characters.
@minrk minrk merged commit faafe3e into ipython:master Aug 2, 2013
@minrk minrk mentioned this pull request Aug 2, 2013
minrk added a commit that referenced this pull request Aug 2, 2013
specify encoding in nbconvert io.open call (finishes #3870).
@minrk
Copy link
Member

minrk commented Aug 2, 2013

Merged here and applied finishing touches in #3879.

@astrofrog
Copy link
Contributor Author

@minrk - thanks for finishing this up, I was traveling today and didn't get a chance to do this sooner.

@minrk
Copy link
Member

minrk commented Aug 2, 2013

No problem - sorry for my impatience, just getting ready to cut the release candidate.

mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this pull request Nov 3, 2014
Fix for converting notebooks that contain unicode characters.
mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this pull request Nov 3, 2014
specify encoding in nbconvert io.open call (finishes ipython#3870).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants