Autocomplete repeats content already in the cell.

[tillahoffmann]

When I use autocomplete, the content that is already in the cell is repeated. The following video probably is the most clear description of the issue. I would have expected only the content after the . to be inserted.

Screen.Recording.2023-02-03.at.16.13.50.mov

Desktop (please complete the following information):

• OS: macOS 13.1 (22C65) on Macbook Pro, 13-inch, M1, 2020.
• Browser: tried both Safari (Version 16.2 (18614.3.7.1.5)) and Chrome (Version 109.0.5414.119 (Official Build) (arm64)).

I've used a clean virtualenv with python 3.10.0 configured with pyenv and pyenv-virtualenv installed with brew. I've removed ~/.jupyter just in case of a funky configuration. I've tried different versions of the notebook package with the same results: 6.5.2 and 7.0.0a11. I'm observing the same issue with jupyterlab==4.0.0a33.

I can not reproduce this issue if I run the notebook in a docker container with the following Dockerfile.

FROM python:3.10.0
RUN pip install jupyter

Some more details below.

$ pyenv --version
pyenv 2.3.11

$ pyenv virtualenv --version
pyenv-virtualenv 1.1.5 (python -m venv)

$ uname -a
Darwin MacBook-Pro.local 22.2.0 Darwin Kernel Version 22.2.0: Fri Nov 11 02:04:44 PST 2022; root:xnu-8792.61.2~4/RELEASE_ARM64_T8103 arm64

$ pip freeze
aiofiles==22.1.0
aiosqlite==0.18.0
anyio==3.6.2
appnope==0.1.3
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
arrow==1.2.3
asttokens==2.2.1
async-lru==1.0.3
attrs==22.2.0
Babel==2.11.0
backcall==0.2.0
beautifulsoup4==4.11.2
bleach==6.0.0
certifi==2022.12.7
cffi==1.15.1
charset-normalizer==3.0.1
comm==0.1.2
debugpy==1.6.6
decorator==5.1.1
defusedxml==0.7.1
entrypoints==0.4
executing==1.2.0
fastjsonschema==2.16.2
fqdn==1.5.1
idna==3.4
ipykernel==6.21.1
ipython==8.9.0
ipython-genutils==0.2.0
ipywidgets==8.0.4
isoduration==20.11.0
jedi==0.18.2
Jinja2==3.1.2
json5==0.9.11
jsonpointer==2.3
jsonschema==4.17.3
jupyter==1.0.0
jupyter-console==6.4.4
jupyter-events==0.5.0
jupyter-lsp==1.5.1
jupyter-ydoc==0.3.1
jupyter_client==8.0.2
jupyter_core==5.2.0
jupyter_server==2.2.1
jupyter_server_fileid==0.6.0
jupyter_server_terminals==0.4.4
jupyter_server_ydoc==0.7.0
jupyterlab==4.0.0a33
jupyterlab-pygments==0.2.2
jupyterlab-widgets==3.0.5
jupyterlab_server==2.19.0
MarkupSafe==2.1.2
matplotlib-inline==0.1.6
mistune==2.0.4
nbclassic==0.5.1
nbclient==0.7.2
nbconvert==7.2.9
nbformat==5.7.3
nest-asyncio==1.5.6
notebook==5.7.16
notebook_shim==0.2.2
packaging==23.0
pandocfilters==1.5.0
parso==0.8.3
pexpect==4.8.0
pickleshare==0.7.5
platformdirs==2.6.2
prometheus-client==0.16.0
prompt-toolkit==3.0.36
psutil==5.9.4
ptyprocess==0.7.0
pure-eval==0.2.2
pycparser==2.21
Pygments==2.14.0
pyrsistent==0.19.3
python-dateutil==2.8.2
python-json-logger==2.0.4
pytz==2022.7.1
PyYAML==6.0
pyzmq==25.0.0
qtconsole==5.4.0
QtPy==2.3.0
requests==2.28.2
rfc3339-validator==0.1.4
rfc3986-validator==0.1.1
Send2Trash==1.8.0
six==1.16.0
sniffio==1.3.0
soupsieve==2.3.2.post1
stack-data==0.6.2
terminado==0.17.1
testpath==0.6.0
tinycss2==1.2.1
tomli==2.0.1
tornado==6.2
traitlets==5.9.0
uri-template==1.2.0
urllib3==1.26.14
wcwidth==0.2.6
webcolors==1.12
webencodings==0.5.1
websocket-client==1.5.0
widgetsnbextension==4.0.5
y-py==0.5.5
ypy-websocket==0.8.2

[RRosio]

Hi @tillahoffmann, thank you for submitting this issue. Would it be possible for you to try downgrading your current version of ipython to something like 8.5.x and see if you still experience this issue?

[tillahoffmann]

Restricting ipython<8.6 appears to do the trick (see below for a locked requirements.txt), thank you! Do yo know why >=8.6 affects the autocomplete?

requirements.txt locked with pip-compile

#
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
#    pip-compile
#
anyio==3.6.2
    # via jupyter-server
appnope==0.1.3
    # via
    #   ipykernel
    #   ipython
argon2-cffi==21.3.0
    # via
    #   jupyter-server
    #   nbclassic
    #   notebook
argon2-cffi-bindings==21.2.0
    # via argon2-cffi
arrow==1.2.3
    # via isoduration
asttokens==2.2.1
    # via stack-data
attrs==22.2.0
    # via jsonschema
backcall==0.2.0
    # via ipython
beautifulsoup4==4.11.2
    # via nbconvert
bleach==6.0.0
    # via nbconvert
cffi==1.15.1
    # via argon2-cffi-bindings
comm==0.1.2
    # via ipykernel
debugpy==1.6.6
    # via ipykernel
decorator==5.1.1
    # via ipython
defusedxml==0.7.1
    # via nbconvert
executing==1.2.0
    # via stack-data
fastjsonschema==2.16.2
    # via nbformat
fqdn==1.5.1
    # via jsonschema
idna==3.4
    # via
    #   anyio
    #   jsonschema
ipykernel==6.21.1
    # via
    #   ipywidgets
    #   jupyter
    #   jupyter-console
    #   nbclassic
    #   notebook
    #   qtconsole
ipython==8.5.0
    # via
    #   -r requirements.in
    #   ipykernel
    #   ipywidgets
    #   jupyter-console
ipython-genutils==0.2.0
    # via
    #   nbclassic
    #   notebook
    #   qtconsole
ipywidgets==8.0.4
    # via jupyter
isoduration==20.11.0
    # via jsonschema
jedi==0.18.2
    # via ipython
jinja2==3.1.2
    # via
    #   jupyter-server
    #   nbclassic
    #   nbconvert
    #   notebook
jsonpointer==2.3
    # via jsonschema
jsonschema[format-nongpl]==4.17.3
    # via
    #   jupyter-events
    #   nbformat
jupyter==1.0.0
    # via -r requirements.in
jupyter-client==8.0.2
    # via
    #   ipykernel
    #   jupyter-console
    #   jupyter-server
    #   nbclassic
    #   nbclient
    #   notebook
    #   qtconsole
jupyter-console==6.5.0
    # via jupyter
jupyter-core==5.2.0
    # via
    #   ipykernel
    #   jupyter-client
    #   jupyter-console
    #   jupyter-server
    #   nbclassic
    #   nbclient
    #   nbconvert
    #   nbformat
    #   notebook
    #   qtconsole
jupyter-events==0.6.3
    # via jupyter-server
jupyter-server==2.2.1
    # via
    #   nbclassic
    #   notebook-shim
jupyter-server-terminals==0.4.4
    # via jupyter-server
jupyterlab-pygments==0.2.2
    # via nbconvert
jupyterlab-widgets==3.0.5
    # via ipywidgets
markupsafe==2.1.2
    # via
    #   jinja2
    #   nbconvert
matplotlib-inline==0.1.6
    # via
    #   ipykernel
    #   ipython
mistune==2.0.5
    # via nbconvert
nbclassic==0.5.1
    # via notebook
nbclient==0.7.2
    # via nbconvert
nbconvert==7.2.9
    # via
    #   jupyter
    #   jupyter-server
    #   nbclassic
    #   notebook
nbformat==5.7.3
    # via
    #   jupyter-server
    #   nbclassic
    #   nbclient
    #   nbconvert
    #   notebook
nest-asyncio==1.5.6
    # via
    #   ipykernel
    #   nbclassic
    #   notebook
notebook==6.5.2
    # via jupyter
notebook-shim==0.2.2
    # via nbclassic
packaging==23.0
    # via
    #   ipykernel
    #   jupyter-server
    #   nbconvert
    #   qtpy
pandocfilters==1.5.0
    # via nbconvert
parso==0.8.3
    # via jedi
pexpect==4.8.0
    # via ipython
pickleshare==0.7.5
    # via ipython
platformdirs==3.0.0
    # via jupyter-core
prometheus-client==0.16.0
    # via
    #   jupyter-server
    #   nbclassic
    #   notebook
prompt-toolkit==3.0.36
    # via
    #   ipython
    #   jupyter-console
psutil==5.9.4
    # via ipykernel
ptyprocess==0.7.0
    # via
    #   pexpect
    #   terminado
pure-eval==0.2.2
    # via stack-data
pycparser==2.21
    # via cffi
pygments==2.14.0
    # via
    #   ipython
    #   jupyter-console
    #   nbconvert
    #   qtconsole
pyrsistent==0.19.3
    # via jsonschema
python-dateutil==2.8.2
    # via
    #   arrow
    #   jupyter-client
python-json-logger==2.0.4
    # via jupyter-events
pyyaml==6.0
    # via jupyter-events
pyzmq==25.0.0
    # via
    #   ipykernel
    #   jupyter-client
    #   jupyter-console
    #   jupyter-server
    #   nbclassic
    #   notebook
    #   qtconsole
qtconsole==5.4.0
    # via jupyter
qtpy==2.3.0
    # via qtconsole
rfc3339-validator==0.1.4
    # via
    #   jsonschema
    #   jupyter-events
rfc3986-validator==0.1.1
    # via
    #   jsonschema
    #   jupyter-events
send2trash==1.8.0
    # via
    #   jupyter-server
    #   nbclassic
    #   notebook
six==1.16.0
    # via
    #   asttokens
    #   bleach
    #   python-dateutil
    #   rfc3339-validator
sniffio==1.3.0
    # via anyio
soupsieve==2.3.2.post1
    # via beautifulsoup4
stack-data==0.6.2
    # via ipython
terminado==0.17.1
    # via
    #   jupyter-server
    #   jupyter-server-terminals
    #   nbclassic
    #   notebook
tinycss2==1.2.1
    # via nbconvert
tornado==6.2
    # via
    #   ipykernel
    #   jupyter-client
    #   jupyter-server
    #   nbclassic
    #   notebook
    #   terminado
traitlets==5.9.0
    # via
    #   comm
    #   ipykernel
    #   ipython
    #   ipywidgets
    #   jupyter-client
    #   jupyter-console
    #   jupyter-core
    #   jupyter-events
    #   jupyter-server
    #   matplotlib-inline
    #   nbclassic
    #   nbclient
    #   nbconvert
    #   nbformat
    #   notebook
    #   qtconsole
uri-template==1.2.0
    # via jsonschema
wcwidth==0.2.6
    # via prompt-toolkit
webcolors==1.12
    # via jsonschema
webencodings==0.5.1
    # via
    #   bleach
    #   tinycss2
websocket-client==1.5.1
    # via jupyter-server
widgetsnbextension==4.0.5
    # via ipywidgets

[krassowski]

Hi @tillahoffmann thank you for getting back on this one. I've suggested downgrading IPython in our triage meeting because there was a substantial refactor of completer in the recent versions (ipython/ipython#13745, ipython/ipython#13852).

The issue you see looks like an edge case when IPCompleter.use_jedi = False and is going to be fixed by ipython/ipython#13943. Based on that (and local reproduction) I believe you have disabled jedi somewhere or use an extension which exposes python_matches; if this is not the case please let me know.

If you are interested in why, see: ipython/ipython#13935 (comment)

[tillahoffmann]

Thanks for the additional context, @krassowski. I did indeed disable jedi because autocompletion was quite slow for me with jedi enabled. I'll keep the downgraded version of ipython for now and/or try enabling jedi again until ipython/ipython#13943 is included in a release. Thanks again.