Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Python 3 on Windows (pyreadline) - expected an object with the buffer interface #842

Closed
pfmoore opened this Issue · 10 comments

3 participants

@pfmoore

This is on Python 3 on Windows, using IPython trunk with the development version of pyreadline from lp:pyreadline (which has Python 3 fixes).

The ipython3.exe console version fails with the following error:

Error in sys.excepthook:
TypeError: expected an object with the buffer interface

Original exception was:
Traceback (most recent call last):
  File "D:\Apps\Python32\Scripts\ipython3-script.py", line 9, in 
    load_entry_point('ipython==0.12.dev', 'console_scripts', 'ipython3')()
  File "D:\Apps\Python32\lib\site-packages\IPython\frontend\terminal\ipapp.py", line 390, in launch_new_instance
    app.initialize()
  File "D:\Apps\Python32\lib\site-packages\IPython\frontend\terminal\ipapp.py", line 301, in initialize
    self.init_shell()
  File "D:\Apps\Python32\lib\site-packages\IPython\frontend\terminal\ipapp.py", line 321, in init_shell
    ipython_dir=self.ipython_dir)
  File "D:\Apps\Python32\lib\site-packages\IPython\config\configurable.py", line 295, in instance
    inst = cls(*args, **kwargs)
  File "D:\Apps\Python32\lib\site-packages\IPython\frontend\terminal\interactiveshell.py", line 113, in __init__
    user_global_ns=user_global_ns, custom_exceptions=custom_exceptions
  File "D:\Apps\Python32\lib\site-packages\IPython\core\interactiveshell.py", line 428, in __init__
    self.init_readline()
  File "D:\Apps\Python32\lib\site-packages\IPython\core\interactiveshell.py", line 1706, in init_readline
    delims = delims.replace(d, "")
TypeError: expected an object with the buffer interface

Looking at the code, it looks like it's a problem of mixing bytes and strings:

            # Remove some chars from the delimiters list.  If we encounter
            # unicode chars, discard them.
            delims = readline.get_completer_delims()
            if not py3compat.PY3:
                delims = delims.encode("ascii", "ignore")
            for d in self.readline_remove_delims:
                delims = delims.replace(d, "")
            delims = delims.replace(ESC_MAGIC, '')
            readline.set_completer_delims(delims)
            # otherwise we end up with a monster history after a while:
            readline.set_history_length(self.history_length)

delims = delims.encode("ascii", "ignore") converts delims from string to bytes, and yet it is only run conditionally. So after this point, the type of delims differs between Python 2 and Python 3. I suspect this is where the problem lies (the "expected...buffer interface" error confirms this, as strings in Python 3 don't expose the buffer interface).

Encoding d and ESC_MAGIC, and replacing the '' literals with b'', works partially but clearly is not the correct fix.

@takluyver
Owner

Hmmm, I think readline.set_completer_delims expected Unicode on Python 3. This might be an incompatibility between GNU readline and pyreadline that we need to work out.

@jstenar
Collaborator

I have pushed a change to pyreadline master that makes get_completer_delims return a unicode string on python3.

ps. pyreadline is now on github if you want to test it

@takluyver
Owner

@jstenar: Will it also accept a unicode string for set_completer_delims?

@jstenar
Collaborator
@jstenar
Collaborator
@takluyver
Owner

Great, thanks.

@pfmoore: Any chance you could test it again with a fresh checkout of pyreadline?

@pfmoore
@pfmoore
@takluyver
Owner

It appears that for various complex reasons, Python doesn't like a Windows terminal using code page 65001. Either change the code page, or, according to this bug report: "The current workaround is to set PYTHONIOENCODING environment variable to utf-8,".

I think it will be fixed for Python 3.3. Anyway, thanks for testing.

@takluyver takluyver closed this
@pfmoore
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.