Skip to content
This repository

Python 3 on Windows (pyreadline) - expected an object with the buffer interface #842

Closed
pfmoore opened this Issue · 10 comments

3 participants

Paul Moore Thomas Kluyver Jörgen Stenarson
Paul Moore

This is on Python 3 on Windows, using IPython trunk with the development version of pyreadline from lp:pyreadline (which has Python 3 fixes).

The ipython3.exe console version fails with the following error:

Error in sys.excepthook:
TypeError: expected an object with the buffer interface

Original exception was:
Traceback (most recent call last):
  File "D:\Apps\Python32\Scripts\ipython3-script.py", line 9, in 
    load_entry_point('ipython==0.12.dev', 'console_scripts', 'ipython3')()
  File "D:\Apps\Python32\lib\site-packages\IPython\frontend\terminal\ipapp.py", line 390, in launch_new_instance
    app.initialize()
  File "D:\Apps\Python32\lib\site-packages\IPython\frontend\terminal\ipapp.py", line 301, in initialize
    self.init_shell()
  File "D:\Apps\Python32\lib\site-packages\IPython\frontend\terminal\ipapp.py", line 321, in init_shell
    ipython_dir=self.ipython_dir)
  File "D:\Apps\Python32\lib\site-packages\IPython\config\configurable.py", line 295, in instance
    inst = cls(*args, **kwargs)
  File "D:\Apps\Python32\lib\site-packages\IPython\frontend\terminal\interactiveshell.py", line 113, in __init__
    user_global_ns=user_global_ns, custom_exceptions=custom_exceptions
  File "D:\Apps\Python32\lib\site-packages\IPython\core\interactiveshell.py", line 428, in __init__
    self.init_readline()
  File "D:\Apps\Python32\lib\site-packages\IPython\core\interactiveshell.py", line 1706, in init_readline
    delims = delims.replace(d, "")
TypeError: expected an object with the buffer interface

Looking at the code, it looks like it's a problem of mixing bytes and strings:

            # Remove some chars from the delimiters list.  If we encounter
            # unicode chars, discard them.
            delims = readline.get_completer_delims()
            if not py3compat.PY3:
                delims = delims.encode("ascii", "ignore")
            for d in self.readline_remove_delims:
                delims = delims.replace(d, "")
            delims = delims.replace(ESC_MAGIC, '')
            readline.set_completer_delims(delims)
            # otherwise we end up with a monster history after a while:
            readline.set_history_length(self.history_length)

delims = delims.encode("ascii", "ignore") converts delims from string to bytes, and yet it is only run conditionally. So after this point, the type of delims differs between Python 2 and Python 3. I suspect this is where the problem lies (the "expected...buffer interface" error confirms this, as strings in Python 3 don't expose the buffer interface).

Encoding d and ESC_MAGIC, and replacing the '' literals with b'', works partially but clearly is not the correct fix.

Thomas Kluyver
Collaborator

Hmmm, I think readline.set_completer_delims expected Unicode on Python 3. This might be an incompatibility between GNU readline and pyreadline that we need to work out.

Jörgen Stenarson
Collaborator

I have pushed a change to pyreadline master that makes get_completer_delims return a unicode string on python3.

ps. pyreadline is now on github if you want to test it

Thomas Kluyver
Collaborator

@jstenar: Will it also accept a unicode string for set_completer_delims?

Jörgen Stenarson
Collaborator
Jörgen Stenarson
Collaborator
Thomas Kluyver
Collaborator

Great, thanks.

@pfmoore: Any chance you could test it again with a fresh checkout of pyreadline?

Paul Moore
Paul Moore
Thomas Kluyver
Collaborator

It appears that for various complex reasons, Python doesn't like a Windows terminal using code page 65001. Either change the code page, or, according to this bug report: "The current workaround is to set PYTHONIOENCODING environment variable to utf-8,".

I think it will be fixed for Python 3.3. Anyway, thanks for testing.

Thomas Kluyver takluyver closed this
Paul Moore
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.