Unicode line crashes traceback print

[ml31415]

Running this code from the ipython shell (2.3.0) and python 2.7.10:

import traceback
from io import BytesIO

def bla():
    ft = BytesIO()
    try:
        raise Exception()
    except:
        from io import StringIO
        traceback.print_stack(sys._getframe(1), None, ft)

bla()

Instead of properly printing the traceback get a TypeError:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-146-757c38f4c237> in <module>()
      5         from io import StringIO
      6         traceback.print_stack(sys._getframe(1), None, ft)
----> 7 bla()
      8 

<ipython-input-146-757c38f4c237> in bla()
      4     except:
      5         from io import StringIO
----> 6         traceback.print_stack(sys._getframe(1), None, ft)
      7 bla()
      8 

/usr/lib/python2.7/traceback.pyc in print_stack(f, limit, file)
    268         except ZeroDivisionError:
    269             f = sys.exc_info()[2].tb_frame.f_back
--> 270     print_list(extract_stack(f, limit), file)
    271 
    272 def format_stack(f=None, limit=None):

/usr/lib/python2.7/traceback.pyc in print_list(extracted_list, file)
     23                '  File "%s", line %d, in %s' % (filename,lineno,name))
     24         if line:
---> 25             _print(file, '    %s' % line.strip())
     26 
     27 def format_list(extracted_list):

/usr/lib/python2.7/traceback.pyc in _print(file, str, terminator)
     11 
     12 def _print(file, str='', terminator='\n'):
---> 13     file.write(str+terminator)
     14 
     15 

TypeError: 'unicode' does not have the buffer interface

Turns out, the last traceback line is formatted as unicode, instead of str like the other lines. Any ideas on that?

[minrk]

What's ft in this case?

[minrk]

Current IPython is 4.0.1, do you see this error with 4.0?

[ml31415]

Sorry, my fault. I updated the reproducer. I got this error, when I was doing tests on a ZODB from ipython. The ft buffer comes from within the library, so not really my choice. I didn't test it on ipython 4.0, just with this somewhat deprecated ubuntu default.

[takluyver]

I guess we're putting some unicode into linecache. We handle all source code as unicode. You can work around this for Python 2 by defining a custom IO class that accepts bytes or unicode:

from io import StringIO
from IPython.utils import py3compat

class MyStringIO(StringIO):
    def write(self, s):
        s = py3compat.cast_unicode(s, encoding=DEFAULT_ENCODING)
        super(MyStringIO, self).write(s)

You can also mostly get by with StringIO.StringIO, but in some situations it will start throwing UnicodeErrors back at you.

Or just use Python 3.

Closing as I don't think we're going to change anything here.

[ml31415]

Well, I surely understand something like, "it's an edge case and no one cares about it" as a reason to close this. But recommending py3 and using unicode as a default, irrespectively of py2 or py3, is some kind of deprecating py2. And that would actually be quite a pity.

[takluyver]

We know that some people care about it, but the possible fixes either break something that more people care about, or add too much extra complexity (I have a fantastic scheme that involves parsing the code twice and playing with the AST, but I really don't want to be responsible for it).