key bindings + modal editing (vim) discussion

[cmyr]

I want to capture some stuff from IRC earlier today, about approaches to vim-mode/modal editing more generally. The kindling question was from @theduke, who asked, should the core have a concept of key bindings?

The more general question is: how should key bindings be handled, and what is the division of labour between core and client, here?

Some general take-aways:

• How important is lossless vim emulation, versus enabling vim-like functionality? For instance: Vim does not let you navigate to the position after the last character on a line. These feels like a hangover from the hardware terminal days. Is this desired behaviour?

• vim mode is itself less important than general 'modal' support, which could be implemented by a state machine that exists in core, consuming raw key events and outputting editor events.

• This feature should exist alongside the current system for handling events; the user would be able to opt-in via some setting.

• The client should (could) have some flag that toggles between raw_input and selector mode; this flag might be set by core in the course of editting; when in raw_mode the client would be expected to just send raw keystrokes. selector mode would maintain the current behaviour.

• integration with IME is tricky, because the client needs to know the core mode in order to correctly dispatch keypresses. For instance (with future vim emulation enabled, and the editor in command mode) if we type ika<return> on a japanese input method, we would want i to be interpreted as 'insert mode', and ka to be passed through to the input method for 'か'. Does this mean that keypresses have to be sync? Some subset of keypresses? Should there be some other mechanism?

• this gets to be more fun if you're in 'replace' mode.

• The lazy option is to not support IME when using the editor modally.

• This raises the larger question of what is the role of core, and what is the role of the client. Contributors who have been more heavily involved in writing clients (@sp3d, @eyelash) are more skeptical of whether core can offer a good solution here.

If anyone thinks I missed anything or has anything else to add, let me know. 😄

relevant: #93 #291

[eyelash]

Great summary, @cmyr! One thing we also discussed was the possibility of moving/duplicating that part of the state machine that decides whether input should go through the IME or not to the front-end but that would bring its own complications.

Also I don't think not supporting IME is even an option. For example on my keyboard layout typing ê or ñ requires an input method because they are two separate key presses each (see this Wikipedia article for more details).

[trishume]

I'm in favour of a solution based on Sublime Text:

Load key bindings from a config file. Key bindings are a mapping from a key+modifiers to a command. Key bindings can have filters attached to them that check things like file syntax, state variables for modal editing, etc...

I personally use a custom kinda-modal key binding set kind of like kakoune except the "mode" is determined by whether my right command and shift keys are pressed. This requires custom keybindings but it doesn't actually require anything fancy and it doesn't exhibit this problem because the front-end knows not to send keys to the IME when ctrl is pressed.

Now, for handling IME, here's some options:

• Synchronously process key bindings and tell front-end whether they should be forwarded to the IME. Possibly with an optimization where this only happens in a "raw mode".
• I don't know how IME APIs work, but what if the letters were normally bound to a send_to_ime command which sent an message back to the front end telling it to send a character to the IME. Do IME's have to receive events inside the GUI API key handler, or can you give them keys later?
• Have an RPC to update the front-end on which key presses will cause a transition to insert mode. This wouldn't work for vim because for ci" the front-end may not recieve the update from the core after c is pressed in time for it to know that i no longer transitions into insert mode.

[palango]

I don't have a solid of the xi-core yet but I would be interested in working on a kakoune mode. For me it feels way more compatible with a multiple-cursor workflow.

I wanted to wait a bit more for the core to stabilize but as the discussion is ongoing this might be good to keep in mind when designing the interface.

[jeanm]

Different modal editing paradigms (vim, kakoune, vis?) use different sets of modes. One could even argue that some vim plugins even add other modes, e.g. vim-easymotion.

With that in mind, is it really feasible to have the state machine live in the core? Perhaps it is, but a lot of thought would have to be put into making it as general as possible.

[linde12]

@jeanm I agree here.

I think making the event handling system pluggable could result in a "vim plugin" and "emacs plugin" etc. I haven't put too much thought in it, but i drew something that i think makes sense.

                                       +----------------+
           +-------------------------> |Frontend process|
           |                           +--------+-------+
           |                                    |
+----------+----+                               |
|               |                               |
|               | <-----------------------------+
|               |
|               |
|Backend process|             RPC EVENT
|               |     {"type":"keypress", key:"h"}            +-------------------------+
|               +-------------------------------------------> |    vim plugin process   |
|               |                                             +-----+-------------------+
|               |                                                   |
+---------------+ <-------------------------------------------------+
                                RPC RESPONSE
                             {"cmd":"CursorLeft"}

A keypress would travel from Frontend->Backend->Plugin and the Plugin would call some action on the backend(which would travel to the & update the frontend as well)

[jeanm]

Another nice thing about doing something along the lines of @linde12's sketch is that the "state machine" logic could then be extracted into its own crate (or crates – one per editor paradigm).

I might be getting ahead of myself, but imagine: if these state machines crates managed to be relatively editor-independent, perhaps defining a common set of backend actions, then different editors (not just xi) and even readline-like crates could use them, which would be pretty sweet.

[cmyr]

@jeanm @linde12 that diagram is approximately in line with how I've been imagining this. The state machine described earlier in this issue should be considered a swappable component, which consumes raw key events and emits various editor actions. It's an open question as to how this should be managed architecturally, e.g. whether or not this should be implemented as a set of components in core or as a type of plugin. Doing it as a plugin adds complexity, but is more flexible: new modes (I use the term mode to refer to an event processing system, e.g. vim-mode, kak-mode) could be distributed independently, which feels nice.

[linde12]

@jeanm Agreed, it would be super cool to have it independent of the editor.

@cmyr I like the idea of having it as some sort of plugin because that might drive the community to implement mode X in a language of their choice. I feel like it would get easier to get started(even with the added complexity of having it as a plugin) if a developer could use a language he/she is familiar with and not have to worry too much about the core.

[purpleP]

@cmyr vim does let you navigate after end of line. help virtialedit

[maxnordlund]

As a vim user, modal editing is a must to be able to switch. That does not mean vim emulation, but being able to edit structured text with just a few keystrokes is paramount.

I don't think this belongs in core as it would make it much more complex to maintain. Especially if you want to support emacs like editing etc. Instead allow the plugin to hijack raw input from the frontend and send commands to the backend.

But there is one thing that would be a real killer feature, the ability to redo/merge high-level commands and not just insertions/deletions. By that I mean a search and replace, auto formatting code or aligning declarations should be automatically applied even when merging multiple clients or when redoing history.

One way to do this is to allow plugins to be able to say, "hey I need to perform som complex operation on the buffer, please give a copy to me". Do that operation, send the resulting diff to core, which responds with it's own diff if needed which the plugin has to rerun the operation on.

An example, Alice and Bob is concurrently editing the document. Alice wants to search and replace foo with bar and asks core for a copy. In the mean time Bob edits some lines, and when Alice responds with what operations to perform core in turn ask Alice to rerun the search and replace on Bobs changes. Once core is satisfied it applies Alices set of changes, potentially blocking Bob.

If Alice operation isn't totally line based it's up to the plugin to either carry enough state to be able to compensate or abort and retry.

Thoughts? Does this sound doable? What else have I missed?

[maxnordlund]

After reading through the https://github.com/google/xi-editor/blob/master/doc/plugin.md file (which I really should have done before posting), I realized some of the ideas is already on the board.

Especially the get the buffer and then stream deltas seems like how I would implement the above complex edit support.

Basically Alices editor plugin uses her copy to compute the edits needed to perform her search and replace, then sends those to core. At the same time the plugin listens for the incoming deltas and re-runs the search and replace on those. Once the plugin is satisfied that the search and replace has been performed, it releases control back to Alice to continue editing. However the plugin can also allow Alice to continue editing, it will probably come down to the command being performed.

My thinking is that when you run a complex command it's not outside the users expectation the the operation will block until complete. But it's always nice if it's done asynchronously.

Edit: s/editor/plugin/

[cmyr]

@maxnordlund: Xi supports resolving concurrent edits from different sources, and all edit operations (such as those performed by plugins) are asynchronous. All plugins are running in their own process, and plugins cannot block the main thread; this is one of Xi's major architectural decisions.

The question of how exactly a vim (or any other modal plugin) should interact with xi-core is still an open question; see my previous comment.

I think that for prototyping, and initial work, it makes sense to work in-stream (that is, building modal editing as a component of xi-core) but I would try and keep the actual event-processing logic (what distinguishes vim mode from kak mode) isolated in such a way that it would be possible to refactor it out of process if we go that route?

[theduke]

The big question is performance.

Model editing, especially akin to vim requires

• intercepting all input
• trying to interpret it
• emit actions, optionally after a timeout (which might just be emitting the same keys again when no action should be taken)

Full vim support is quite challenging due to:

• regular key bindings even in insert mode (eg many people bind ;; or jj to ESCAPE to exit the insert mode. this requires intercepting the j, waiting for TIMEOUT ms to see if another j comes in, if so, exit the insert mode, if no, just emit the j into the buffer.

• nested keybindings: for example you can have key mappings for both gc and gcx. Again, after the user enters gc, we need to wait for TIMEOUT ms to see if the user enters x, ant then either execute tha mapping for togcorgcx`

This would require some special kind of subscription so that only the plugin receives all input and nobody else.

Also it's quite performance critical since the plugin first needs to receive, interpret and respond to the input before the user sees a change. Not sure if plugins will be fast enough here.

[maxnordlund]

As @theduke says, the vim plugin needs to be on top of the food chain here as it converts keypresses into actions. This is true for any key mapping plugin, not just a plugin that registers a couple of keyboard shortcuts.

This is why it can block, by basically stop processing keypresses until some operation finishes. This may be orthogonal to a responsive frontend. I.e. moving around the cursor, scrolling would work, but edits/shortcuts doesn't.

In vim there are two timeouts, one very small (10 ms) that's used for interpreting escape codes sent by the terminal, and one larger (300 - 1000ms) that combines multiple discrete keypresses into one command. Like the gc / gcxexample above.

So the question then is, who's responsibility is it to listen/consume keyboard events, or more generally, any sort of user input inside the editor window? I would assume the frontend is the one to first receive those, but should they be sent to all plugins? Just some according ot predefined keyboard shortcuts? All of them at once?

---

Edit:
I just thought about one solution to the timeout issue. Unlike vim one could require all keyboard shortcuts to be unambiguous. Basically no overlap, so the gc would be illegal. It does make some stuff a bit tedious, but I can't think of two commands that overlap except my own.

I should also say that I would love to start writing a vim-mode plugin when it's starts to be possible.

[theduke]

Unlike vim one could require all keyboard shortcuts to be unambiguous.

That would prevent key bindings in insert mode. I could live without that feature though, especially for a prototype. But I did talk with raph and cmyr on IRC once and they said the timeout thing wouldn't be such a big problem in core.

[maxnordlund]

If it's not a problem then hurray! But I don't understand how unambiguous keyboard shortcuts precludes bindings in insert mode? At least for does with modifiers it would be fine, like how it's right now in vim. E.g. <C-=> resizes windows, or something on meta.

[hh9527]

i think xi-vim-plugin should be a stangalong process between frontend and the xi-core, the modal/state shoud stay in vim-mode-plugin, it can tell the frontend whether receive normal input event or just keyboard event(auto disable IME for the case)

[purpleP]

I'll give my two cents here.
First of all the easiest and cleanest architecture in this case I think would be to broadcast all key events to all plugins and they would react to them.

In this case default keymap would be just kind of preinstalled plugin that would call for example when i key is pressed insert_char('i', position). That's what emacs does if we forget about interprocess communication overhead that xi has. But what emacs does the right way is that it creates correct layers of abstraction.

First layer is the layer that describes how to work with the buffer: move cursor, insert and remove characters etc. Then it describes layer that can map key events to different functions. And then it uses this two layers to bind keypresses to function which modifies the buffer by inserting characters. There is also a layer that describes working with windows (splitting, moving, resizing etc).

If this isn't possible because of responsiveness reasons I think plugins could describe their state-machine in some form.

{
  "normal": {
    "i": "vim.to_insert_mode",
    "h": "core.move_cursor_left",
  },
  "insert": {
    "i": "core.insert_char", "i",
    "h": "core.insert_char", "h",
  }
}

And they should somehow report what state they're in so that core could call appropriate functions via RPC. This would allow to detect conflicting keybindings right at the start of the editor. This would reduce inter-process communication, but still would allow plugins to react to arbitrary keypresses and implement their own state machines in plugins themselves instead of xi-core.

As a vim user I can say, that almost no vim/emacs user would use this editor unless it would provide all the features (or a way to emulate them) that they use in vim/emacs.
What I mean is that vim-like keybindings are useless by itself. Almost no vim user use intellij vim emulator or visual studio code vim emulator because vim isn't just about text-objects and modes, it also a way of efficiently switching between buffers, splitting windows and stuff like that.

So IMHO unless xi would have extensibility at the level emacs has where basically all vim features are emulated in its evil-mode plugin (if not than it's at least possible to do so quite easily) you can just not waste time trying implement vim experience at all.

Actually I thought about this for a while and I don't see any problems with keybindings, plugins and state machines etc. It's all very much solvable problems (I just show at least two ways of doing that). What bugs me is how to enable plugins to make their own UI elements like autocomplete popup window, window to show docstrings, command-line etc...
I think the most perspective solution is to have some kind of core part that would describe what widgets are possible to create and what is possible to do with them (like resize window, split window etc) and force UIs to implement them.

[maxnordlund]

As a vim user I can say, that almost no vim/emacs user would use this editor unless it would provide all the features (or a way to emulate them) that they use in vim/emacs.

That's a very broad generalization. As a vim user I don't expect, or want, the full vim/emacs feature set. What makes vim great is it's modal editing system, not all the stuff that has creeped in over the years. A vim plugin should be focused on just that, being able to quickly and efficiently edit text.

Besides, you're not ready to switch to a new editor with that kind of attitude. Full feature parity with vim basically means neovim, or remacs for emacs.

What I mean is that vim-like keybindings are useless by itself. Almost no vim user use intellij vim emulator or visual studio code vim emulator because vim isn't just about text-objects and modes, it also a way of efficiently switching between buffers, splitting windows and stuff like that.

Again with the generalizations. I did install the vim bindings for intellij and they sorta worked. But because they didn't think of supporting modal editing from the get-go it's always going to be a hack with the resulting glitchiness. The reason evil-mode is so feature complete is because it's so old and had time to implement the more obscure stuff.

Vim has tacked on window management, quick fix/location list, folding of blocks, a bunch of C specific stuff (:make, special indentation handling, etc.) amongst others. Even the scripting language is a jumbled mess, if you ever had to implement something in then you know how hard it can be to get right.

Xi has a golden opportunity to break free from such legacy and get things right from the start. But expecting full feature parity from the start is unrealistic, and I stand by my opinion that if you're not ready to compromise, then you aren't ready to switch. I rather see xi going in the direction of spacemacs witch tries to combine the strength of both emacs and vim instead of being yet other clone.

[purpleP]

@maxnordlund

As a vim user I don't expect full vim/emacs feature set

Neither do I. I'm saying that ability to implement all of them would indicate that architecture have been done right.

you're not ready to switch to a new editor with that kind of attitude

Yes, I am. I've tried to switch to emacs with evil-mode and it mostly worked out well.

I did install vim bindings for intellij and they sorta worked.

So did I, and that's the reason I stopped using intellij and VS code. Vim bindings sorta works, but vim is also about efficient window management and customization. And because both intellij and VS code use tab as a metaphor for open file they can't provide efficient window management.

the reason evil-mode is so feature complete is because it's so old

I would disagree on that. I think the reason it's so feature complete is first of all because emacs architecture have been done right and it provides means to implement vim with all of its behavior inside emacs. Evil-mode authors just used this means.

I rather see xi going in direction of spacemacs

Well I would rather not. I'd rather see xi going in direction where plugins could implement whatever text editing is possible, be that vim-emulation with modes, window-management and command line or even emacs-clone.

What I'm trying to say is that it would be better to make architecture flexible enough to allow that rather than creating ad-hoc solutions that partially work.

As to my generalization - I have a reasoning for it. For now both vim and emacs have practically all the features that one can require from text editor. The can show errors, search and replace with regexes, syntax highlighting, window-management, autocomplete etc.

How many vim/emacs users would switch to an editor, the only benefit of which is the ability to edit 300 MB files, while they would loose a lot of features? I'd say not much.

It would be practically a sin to spoil the excellent core and ideas about async plugins and CRDT with bad architecture that isn't flexible enough to provide advanced use cases.

We have opportunity to learn from mistakes vim/neovim/emacs had and create an editor that is better in every way, which would provide ability to do more. For example in neovim there have been a long discussion about that neovim-core should provide a way for plugins to create and manage their own widgets. If this would be possible, suddenly external plugin that makes command-line for emacs or vim would be possible. Or a plugin that show search results and allows to mirror edits in them into actual files. Or showing function arguments and autocomplete at the same time (right now vim can't do that and neither emacs).

[purpleP]

@maxnordlund I think what I've described is more or less the same what @cmyr and @linde12 are saying.

• State machines should rather be in plugins, not in the the core.

• To make UI more responsive the travel of keypress instead of looking like Frontend -> Backend -> Plugin -> Backend - Frontend could look like Frontend -> Backend (uses plugins manifests to know how to dispatch keypress) -> Plugin/Backend function -> FrontEnd). This would provide a way for backend to invoke it's own functions directly if there's no need to call plugin function.
  This would require some kind of DSL to describe state transitions and keypresses, but this is solvable, Steam Controller already solved that.

• Plugins should have ability (at least a limited one) to provide custom widgets and manage keyevents in them.

For example autocomplete could be done conceptually as a plugin that would subscribe to char_inserted_to_buffer, check if there is something to show in autocomplete and show it in custom widget which would have it's own state and keybindings (like tab would select next option, space select the current one and close autocomplete etc).

[maxnordlund]

How many vim/emacs users would switch to an editor, the only benefit of which is the ability to edit 300 MB files, while they would loose a lot of features? I'd say not much.

You're right, it would be hard to convince a die hard fan of either vim or emacs to switch. But perhaps that shouldn't be the goal.

… Or showing function arguments and autocomplete at the same time (right now vim can't do that and neither emacs).

You mean like this?

Using YouCompleteMe vim plugin in a JavaScript file

---

Regarding the architecture, I think I know what you're getting at. Keeping state in the plugins is a given in order to keep core small. But I think broadcasting all user input to all interested plugins is the way to go, allowing them to determine if they're relevant or not. Modal editing á la vim basically means having multiple key maps, and the currently selected mode determines which is relevant. That seems hard to keep inside core without also keeping state and/or making it way more complex.

As for custom widgets, it's definitely needed. But shouldn't that be up to the frontend? A GUI vs TUI would have vastly different capabilities, right?

But there has been talk about having core exposing some sort of inter-plugin RPC, and that would allow the frontend(s) to expose a common API for widget control (and line gutter/status bar/tooltip/…)

Actually, having core, or something equally official, define a set of common API:s would make the ecosystem much more coherent and make most plugins just work™️ together. A bit like syntastics/neomakes all makers, or Atom/Sublime linters.

[maxnordlund]

As an aside, I'm sorry about the harsh tone, generalizations do trigger me but that's no excuse for being impolite.

[purpleP]

@maxnordlund I don't see anything impolite in you words. I care only about seeking truth in the argument.

You mean like this...

No, not like this. Like intellij does it (can't provide a screenshot right now, but you seem to have tried jetbrains products so you probably know what I'm talking about). Where there is two windows, one with autocomplete and one showing what positional argument you entering right now with name and the type of the argument.

it would be hard to convince a die hard fan of either vim or emacs to switch

Well, yeah if we're talking about "fan" meaning the person that's denies actually truthful statements like that emacs architecture is better than vim and it's a lot easier to customize emacs etc (this goes both ways), then I would say this is practically impossible.
But I'm not talking about "fans". I'm talking about people that use their editor to the full extent like people that use Vim or Emacs. editor that can edit 40 MB files easily isn't big market share I think.

But shouldn't that be up to the frontend? A GUI vs TUI would have vastly different capabilities, right?

Yeah, I gaved it a thought. If it would be up to a frontend it would create a bunch of incompatible plugins that work with just one frontend. While this can be done, I think the better way would be to establish some common ground that every plugin could use.

And GUI and TUI doesn't have that much different capabilities. The only really useful differences I know are ability to use different fonts in different places (emacs-gui uses this in markdown) which I wouldn't consider a major improvement. And the other one which no editor use right now (aside from Word etc) is ability to render things like math formulas (think about emedding tex math inside of markdown) as math formulas when they aren't edited. (I don't consider sublimes minimap as a useful feature. You could jump into places much faster with search, tags or marks). It's possible to create almost all GUI widgets in TUI like buttons, text fields, checkboxes (basically everything aside from drawables)

Because the goal of this project stated in readme is to use best technologies for text editing I think using best architecture to provide extensibility, that no other editor provides right now is also can/should be a goal. For example emacs tries to show function docs and arguments when autocompleting, and because emacs doesn't provide a way for plugins to define their own widgets they do it by modifying the actual buffer and then redoing the changes. Vim has a problem where popup menu is refreshed by removing it and adding it, which created flickering. That's what I mean when I say that we can learn from their mistakes and do better.

For example I think that one shouldn't create a TUI for a specific editor, but instead do a general purpose library that can create as many widgets as possible (not right away of course) (maybe curses already does that I don't know) in TUI (obviously 3d effects wouldn't be possible) and a way to incrementally update them instead of redrawing everything every time one widget changes (which seems to be the problem with vim in some cases). And only based on that layer build a TUI for any editor. Which doesn't seems to be the way vim or TUI for this project works.

But I think broadcasting all user input to all interested plugins is the way to go

Well, so do I. I've just provided one possible way to optimize if the round-trip delays would be unacceptable. The other way, by the way, although I'm not sure how possible this is in compiled language is to use C foreign function interface as a way for performance critical plugins to interact with core. Almost every language can call C functions and provides a way to call function defined in this language from C. Because rust is compatible with C this may be possible, although I'm not sure how the core would know which plugins expose what functions at runtime. Just a thought...

[rtfeldman]

Just wanted to note that to me, it makes a huge difference if a given editor supports vim-like features with or without vim macros. If there's no good support for macros in an editor I'm using, I'll switch back to vim just long enough to write and use the macro I need, because the overall time savings often far outweigh the time cost of temporarily switching editors.

I want to call attention to this use case because a lot of editor plugins either never ship it, or end up shipping a frustratingly buggy half-baked implementation.

It seems like the type of feature that is much, much easier to support well if you design for it up front.

[jakalope]

Hey all, I've started building a Vi-state-machine in Rust, called ViXi. As the name suggests, the plan is to incorporate it into Xi either as a plugin or as a fully fledged Xi-core feature. Thanks @cmyr for pointing me to this discussion -- it clearly asked the same question I've arrived at in my code, namely what is the interface between Xi and a Vim-like op/remap feature?

As I read all of these posts, I'm tending to feel that the state machine needs to be part of the core and any attempt to make a Vim-compatible set of mappings can live in either a config or a plugin.

[jakalope]

After some thought, I agree with the near consensus that Xi-core in principal should delegate any modal code to a separate plug-able interface. I'm concerned that making such an interface as generic as the existing RPC calls or allowing multiple such plugins to run simultaneously might cripple the functionality.

In general, Xi-core must know how to perform some basic set of instructions in order to communicate with a plugin on any level. A lot of Vim's power comes from a small subset of its principal functionality. I'm confident such a subset can be implemented in the core and a lot of additional functionality can be expressed as sequences of these functions.

For example, take ciw (change inner word). First, c can be expressed (granted, without . repeat capability) as diwi. Thus c need not be expressed directly in Xi-core but can be handled as a combination of other commands, much like RISC assembly is a minimal subset of necessary CPU instructions.

[jakalope]

What do others think about having an input plugin type that sits between a frontend and the core?

[purpleP]

@jakalope Well, having vim emulation without repeat operator would be useless. I mean it's a key feature. I think having event driven architecture would make all this more customizable.
For example vim emulation plugin could emit mode changed event and front end would do whatever it wants with it.

[cmyr]

@jakalope Sorry for not being a bit more involved in this issue.

The idea of having an 'input plugin' receiving keystrokes is approximately our current thinking; the question is whether or not this is actually a plugin, in the xi sense. See this comment #302 (comment).

What we're interested in building is a general system that supports handling of raw key events; and then on top of that building (say) a vim-mode.

So what do we need to do to get this started?

• we need clients to be able to send raw events
• if we want to handle IME, we need to have this be a synchronous call, with the return value indicating whether or not we are still sending raw events
• we need to figure out how we exit insert mode, e.g. we need to implement cancel_operation and make that synchronous as well?
• we need to add a code path in xi-core that passes raw events to a given handler (e.g. the vim handler)

If someone wants to work on this stuff, I'm very happy to help out and offer whatever guidance I can, including adding client support to xi-mac if that's useful.

also (@jakalope) a good place for discussion is often the #xi channel on irc.mozilla.org.

[yqrashawn]

What makes vim great is it's modal editing system, not all the stuff that has creeped in over the years. A vim plugin should be focused on just that, being able to quickly and efficiently edit text.

For me, I think the emphasis here is to implement a home-row editor. The vim-like window, split switching is not a must, but user being able to switch them in home-row is a must.

[maxnordlund]

The thing is that the home row is different for each keyboard layout (surprise surprise), so focusing on that would by definition not quite work. As an example, my home row is asdfghjklöä'.

Related to that, it's really annoying when a program doesn't take non-english keyboard layouts into account, like using brackets, colon, semi-colon, etc for shortcuts. All of these already require modifier(s), which means that for example SHIFT+] is impossible for me to type. What I'm trying to get at, allowing the user to change any and all keybindings solves this, so lets not focus on a specific set of shortcuts right now.

[yqrashawn]

@maxnordlund That's exactly what I'm saying. User must be able to call any function (interactive one) with their home row.

As for the timeout issue. Emacs evil-mode don't support that, but keychord-mode do support that feature. So maybe we can take them separately.

[SamHasler]

Just supporting movement and editing keys for switching modes is just scratching the surface. See this comment: https://news.ycombinator.com/item?id=15599880 for how undo handling affects the vim mode in VSCode. I've heard it said that there can be a kind of uncanny-valley with vim emulation, where it's fine up until you try to use an unimplemented feature or behaviour that you're familiar with to the point that it's in muscle memory. Having a list of where such features/behaviors are would be helpful in deciding how to layer the application and what the boundaries and concerns of different parts will be. I expect it would be easy to make such a list from looking at how other open source text editors handle vim emulation. ᐧ

…

On 2 November 2017 at 09:31, yqrashawn ***@***.***> wrote: @maxnordlund <https://github.com/maxnordlund> That's exactly what I'm saying. User must be able to call any function (interactive one) with their home row. As for the timeout issue. Emacs evil-mode don't support that, but keychord-mode do support that feature. So maybe we can take them separately. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#302 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADUBWHMUBvSSrWkQmJLz1CA9pZqcbcPks5syYwOgaJpZM4NelSw> .

[zacps]

@cmyr I'd be interested in working on this once there's a consensus on how to best do it.

[cmyr]

@zacps Is there anything in particular you're interested in? Like core support, or implementing an actual modal editing system? More guidance on this issue should be available next week, when @raphlinus is back at work.

@jakalope I responded to you on IRC but not sure if you saw it; in any case logs are here https://botbot.me/mozilla/xi/.

[zacps]

I'd be interested on working on core support, I think I might actually be a bit too busy the next three weeks or so. If it's still free then I'll jump at it.

[vitiral]

just to give my 2c as a vim user and light follower of this project, feel free to ignore.

It seems to me like the vim/emacs/etc commands could/should be supported in core, while the vim keypresses themselves should be ignored.

For instance dtf in vim is "delete untill f" and 2dtf is "delete untill 2nd f". I could see having a large number of these kind of "commands" being supported to help with editing speed for multiple frontends. In my understanding, most editors have "commands" like this, they just have different ways of going about how the user executes them.

However, while these commands could help with speed, it should be considered far more important to make it easy for plugins to be able to "emulate" commands from the frontend's perspective. I.e. chaining "core-commands" in a macro shouldn't be any more difficult than chaining "plugin-commands"

I'm not sure I'm 100% sold on needing to do too much of this in core, especially related to core being responsible for handling+repeating events. I feel like the frontend should always be responsible for storing+repeating a chain of commands, and the command API is responsible for making that "repeatable".

Maybe a good use case is dtn or "delete until next", "next" being the next item in the search.

Regardless of whether core or plugins support this "next" command or even the "delete/until" commands, it should be possible to express this as a single command to core and have core execute it. Something like:

• what am I doing? I'm deleting text.
• how much text? I need to use the "until" and "next" chain to find that
• I now have a complete command which I execute.

Some of these may be able to be answered directly in core and some may need to be answered by plugins -- but the design should require that it doesn't matter (except obviously you can get performance improvements by supporting some things in core).

[dm319]

This is a little bit meta, but what would we want the 'plugin' to emit?

I think 'vim-keybinding' is a misnomer - it's really an interactive 'vim-language', which needs to be processed by an interpreter.

If we look at it that way, then what should the interpreter emit? What's the assembly language of text editing? Are we talking keypresses? (though that will need to be interpreted). Diffs? Or some kind of assembly language that has a core set of basic text-editing functions?

[euclio]

Something to consider as well is that many Xi RPCs don't quite map to vim behavior. For example, in implementing my own Xi frontend, I've found:

• Moving down on the last line in vim will keep the cursor in the same cell, while in Xi it will move to EOL.
• Moving right a word in vim will put the cursor on the first letter of the word. In Xi, it will move the cursor to the space in front of the word.

Should the core provide another set of RPCs to match the vim semantics, or should the frontend/plugin compose motions to try and emulate them (which might not be fully possible).

[cmyr]

@euclio I'm leaning towards the second, but this will involve rewriting some stuff. For the first example, #826 is related; we can add 'movement units' that correspond to things like 'start of word' and 'end of word'.

The 'move to EOL on down in last line' feels like it's mostly a mac custom, maybe? I'm not totally sure how best to think about this, but I am thinking a lot about key bindings and input in general right now, and I'll keep this in mind..

[mninja]

I'm currently working on a vi-like frontend for xi and would like to echo some of the thoughts brought up in this issue.

Mainly, I don't think it's scalable for xi-core to handle modal editing or vi commands, unless the intent is for xi-core to be a vi clone and for all frontends to support it. Here are some of the problems I see, most of these have been brought up already:

• xi-core itself will be more difficult to maintain
• Introduces more synchronization points between the frontend and core (cursor style, current mode, command buffer...)
• How do you deal with buffers not for editing text? e.g. markdown preview, embedded terminal, settings panel, etc. I would still want vi bindings and commands to be possible, but it doesn't seem like xi-core should know about those buffers or handle their input.
• xi-core would need to understand all sorts of input events, not just "here's some unicode". Should shift-h be insert H, keydown: shift + keydown: h, keydown: h (shift: true)? How does the frontend differentiate between when it should just send unicode versus raw key input?
• How do you determine the subset of vi to support?

Note: the above aren't meant to be unanswerable questions and answers have already been suggested, I just think that having to answer them is the problem.

Instead I would suggest the following:

• xi-core adds primitive events to make vi commands easier to implement. For instance, a ViMoveLeft that does not go to previous line as MoveLeft does, InsertNewlineBefore for O, etc. I think composition should be avoided as it might cause strange issues with replaying/undo points, unless xi-core has a way of handling this.
• A xi-vi-mode could be a library or plugin that takes input events and potentially translates them into equivalent xi-rpc commands.
• A frontend can use the xi-vi-mode or not, as appropriate. Since xi-vi-mode doesn't automatically talk to core, the frontend could reuse it for buffers/views core does not know about.

What's nice about the first point is even if xi does decide to subsume vi responsibility or ship it to a xi-plugin instead of letting the frontend do it, those primitive events would still be needed. I'm trying to experiment with a local xi-core fork to try out this idea. I could make a PR for these new primitives if there's interest.

[cmyr]

@mninja i've seen this and intend to respond, will try and get back to you in the next day or so. :)

[tmandry]

I would be concerned if changing particular how an editor command behaved required a change to the primitive in xi-core.

Talking about vi emulation specifically, these nuances of how a particular motion/action works in specific contexts are what I've seen many vi emulations wrestle with. These are what create the "uncanny valley" that someone referenced earlier. In my opinion, tweaking this has to be easy and light-touch to succeed.

(Even if your goal isn't perfect vi emulation, I think it's a great proxy for the kind of flexibility we would eventually want from xi-core.)

To make this possible, actions should be as generic as possible and composable, maybe even with support for conditional blocks and expressions. I think executing multiple actions atomically is critical to making this work, and don't think it would be difficult in theory to implement.

[mninja]

I'm not sure that changing primitives would be necessary. Are there so many different possible meanings of "move left" that it's not feasible to provide all of them as primitives? Or do you mean if someone wants to make arbitrary changes to their keybinds?

I think composing make sense overall, but I'm not sure how you would compose something like "move one character directly down, including word wrap".

[maxnordlund]

I believe what is needed is some form of transactions. "Change these ranges to XYZ and move the cursor(s) to these ranges"

[cmyr]

Firstly: I think the xi-vi-mode component is very similar to the approach I've been thinking about, where raw input can optionally be handled by some class of 'input plugin', that handles mapping events and tracking other mode-specific state.

Secondly: This touches on one of the major untouched corners of the project, which is keybindings and key mapping in general. To date, we've really punted on this question, but we can't do that forever.

One way to think about this is, "how do we turn raw input events into editor actions"? Currently we've just picked one model, based heavily on MacOS/Cocoa/AppKit conventions (which are similar to conventions on windows and desktop linuxes), and we've just baked those into the editor. In my ideal future world, all input handling, including the current default handling, is implemented in a modular way, so there is a "handle raw input" step before the "send editor events" step.

As I haven't dug deeply into this, it's possible that this is just too difficult for various reasons, but it remains my ideal.

I think adding new primitives makes sense in some cases, but I think that composition is probably going to be something that we want longer term.

In terms of the "move down, respecting word wrap"; this is just what the move_down RPC does, currently; the commands that are about relative modifications to cursor position are all intended to account for visual, not logical lines. This is the sort of stuff that does need core support, in some sense, because you need access to view state.

I'm not sure if that answer is helpful, so let me know if I can clarify anything in particular?

[mninja]

This clarifies a lot, especially with regards of where xi is intended to sit on this. The approach makes more sense in that light but I'm still wary, especially when it comes to handling buffers that don't represent plain text.

Are there any short term plans for this? I'm currently hacking with a local xi-core that has a couple of extra movement primitives which feels like a quick win.

[cmyr]

There aren't any immediate term plans. This ties into a bunch of other questions that we've had about the overall architecture of the project, and I think we'll really need a sense of that bigger picture before we can start implementing this.

That said, if you would like to PR new movement commands, we're totally receptive to that.

[rtfeldman]

That said, if you would like to PR new movement commands, we're totally receptive to that.

It's worth revisiting @palango's comment about kakoune in this context.

Kakoune has a ton of overlap with vim (modal commands, movement, etc) but it uses multiple selections as a primitive.

My intuition is that implementing movement commands that facilitate nice Kakoune support would be suficient primitives for nice Vim support as well, but not the other way around.

[ficoos]

Just my two cents,
IMHO xi-core shouldn't handle any key events at all.
From what I gather, xi-core is meant to synchronize edits from multiple sources asynchronously.
The UI isn't any different from a plugin. It either generate changes from external events or
as a reaction to a xi event.
The UI reacts to keyboard events creating changes that are sent to the xi-core.
The xi-core broadcasts about changes in views and UI reacts by drawing those changes on screen.

 keyboard -> UI: translate to commands -> xi-core -> UI: translate to buffer changes -> screen

So the actual place to put the actual modal handling is in the translation layer between the keyboard and the actual generate RPC messages.
This is clearly in the UI domain.

This doesn't mean that all interfaces have to reimplement modal handling.
You could have an abstract modal input handler that accepts abstract input and fires xi-core changes.
The UI will collect the input from the textbox, terminal, virtual keyboard and feed it to the common handler.
The view is unchanged since it is disconnected from the input by design anyway.

input -> UI: translate to abstract input -> modal-core: translate to commands -> xi-core -> UI: translate buffer changes -> screen

As for complex operations that are currently hard to implement atomically with the current xi-core API.
This is orthogonal to modal editing and needs to be solved on it's own.

As a related note, you could have UIs for people with motor disabilities that use something other than a keyboard to interact with the editor.
You want them to be able to use all the functionality and bind "hotkeys" to whatever input device they use.