start using new codemirror completion

[Carreau]

Should start fixing as you type completion.

This use the brand new completion-hint introduced by codemirror short after I wrote the completer.
it automatically handle the "as you type".

The only annoying things I see and I'm not sure how to handle :

• We prevent event "downarrow" from propagating when the cursor is at the last char of codemirror, whic prevent the selection of next item in completer.
• esc dismiss completer and escape to command mode (but not always)

TODO before merge :

• remove path that say codemirror is up-to-date

[Carreau]

there are probably also some aligning bug to send upstream to codemirror when completing near edge of screen.

[Carreau]

This need a (one line) patch to CM, to work, and (should) fail gracefully on normal version. You cannot press tab to complete to the longuest prefix of completion without the patch.
It also trigger completion on . (dot) now.

[Carreau]

So IPython.Completer is gone now. I think this could affect @gibiansky, @BayardRock and in julia I suppose @malmaud that deal with codemirrors modes IIRC.

Just to keep you guys updated why this. Current completer was completely broken (preventing user to type) So I had to re-write most if not all of the things.

First thing, if you extend CodeCell.options_default.cm_config.extraKeys with something like "'$'": "completePassthrough" (yes single quote in double quote) it will invoke the completer on $ and insert $. It interesses some of you right ?
It is also much more easier to re-bind Tab to another shortcut.

Second <Tab><Tab> now with codemirror Master complete to longuestCommonSubstring, not only dealing with percent in %magics I think this should be tested. It fallback to pick current result on CM < 3.21.

Third, completion does not include analysis of surrounding tokens anymore for the time beeing, I'll need to rework some of the internal to allow that again.

@ellisonbg I still need help when completer is dismissed with esc not to switched to command mode.

This is a lot of change close to release but I think it is needed and the experience is much better that current and old completer.

This also layout ground to allow things like modification of completer like IfSharp does (icon in front of completion + tooltip for currently highlighted completion)

Please test and feedback ! Thanks !

[gibiansky]

@Carreau How does this change the messaging protocol, if at all? The only thing IHaskell did was use the complete_request and complete_reply messages - is this just a client-side change, or are those messages somehow affected?

[Carreau]

@gibiansky only client side. I just thought you were patching to trigger completion on other keypress than tab.

[Carreau]

Re-added options to complete from surrounding tokens. Deactivated on CM <= 3.21.1 because of bug.

[ivanov]

I've also been diving into CodeMirror / IPython interaction, so just adding myself to this PR to keep track of it.

[Carreau]

needs codemirror/codemirror5#2228

[Carreau]

With master codemirror this should mostly work and be (depending on use cases) being equivalent to previous completer on 95% of cases.

The few edge cases I can think of are :

• longuest common substring (lcs) is not at the beginning of the completions:
  • pandas.sandbox.<tab> show pandas and sandbox, lcs is and so tab once more complete to pandas.sandbox.and. In the other end you cannot force the lcs to start at beginning of word as you want the lcs of %matplotlib and matplotlib to be matplotlib and not empty string. Problem would be worse with subsequence instead of substring, but calculating the LCS of more than 2 string is hard one could calculate the LCS only on the union of from:to: range by extending the current completion with adjacent token but it's a small project on itself.
• (current) lcs might not make sens on completion that do not have the same from:to: range and might be overzealous at replacing something it shouldn't, or not replacing it were it should have. I have no real world example, but I suppose we could come up with cases were numpy.random.norm<tab><tab> would complete to numpy.normal (dropping random, even maybe numpy) or to numpy.random.numpy.random.normal.

There are other annoying related things due to the fact that completion from the kernel do not come with a from:to: range and I use heuristics, so I'm not sure it is worth trying to that right now.

[Carreau]

@ivanov, do you want to bind completion to Ctrl+N and check that esc does not interfere with vim mode ?

[Carreau]

Remove this line before merge. Use only when testing with dirty submodule and a master CM.

[Carreau]

Painfull rebase 😡.

[jdfreder]

Looks like this needs another rebase 😦 However, if I were you I would wait to rebase till #5297 gets merged, because it lightly touches the completer.

[Carreau]

Sure, anyway this is 3.0 material.
I'll do that when I make a break in writting my dissertation.

[Carreau]

closing in favor of issue #6031.