Shallowly switching between libmodal modes

[mawkler]

Hi agan! I'm trying to switch between multiple libmodal modes. However, exiting libmodal seems to require one <Esc> press per mode switched. My expected behaviour is that pressing <Esc> would immediately take me to Normal mode, just like how all Vim modes work. Do you agree? If not, is there some other way to exit libmodal? There doesn't seem to be a require('libmodal').mode.exit() that I can create a mapping for 🙂

To reproduce:

local M = {}

M.mode1 = {
  m = function()
    vim.notify('Mode 1')
  end,
  n = function()
    require('libmodal').mode.enter('My mode 2', M.mode2)
  end
}

M.mode2 = {
  m = function()
    vim.notify('Mode 2')
  end,
  n = function()
    require('libmodal').mode.enter('Mode 1', M.mode1)
  end
}

vim.keymap.set('n', 'M', function()
  require('libmodal').mode.enter('Mode 1', M.mode1)
end)

1. Press M to enter Mode 1
2. Press m: "Mode 1" is notified
3. Press n
4. Press n to enter Mode 2
5. Press m: "Mode 2" is notified
6. Now I have to press <Esc> twice to reach Normal mode

[Iron-E]

Exiting libmodal seems to require one press per mode switched. My expected behaviour is that pressing would immediately take me to Normal mode, just like how all Vim modes work. Do you agree?

My thinking about it is like this:

• Entering a libmodal mode is like entering Insert, from which you press <Esc> once to return to Normal.
• Entering a sub-mode (i.e. a libmodal mode inside a libmodal mode) is like :h i_CTRL-O, from which you press <Esc> twice to return to Normal.

If not, is there some other way to exit libmodal? There doesn't seem to be a require('libmodal').mode.exit() that I can create a mapping for

I opened a PR for this :)

local M = {}

M.mode1 = {
  m = function()
    vim.notify('Mode 1')
  end,
  n = function(self)
    require('libmodal').mode.enter('My mode 2', M.mode2)
    self:exit()
  end
}

M.mode2 = {
  m = function()
    vim.notify('Mode 2')
  end,
  n = function(self)
    require('libmodal').mode.enter('Mode 1', M.mode1)
    self:exit()
  end
}

vim.keymap.set('n', 'M', function()
  require('libmodal').mode.enter('Mode 1', M.mode1)
end)

[mawkler]

Calling Mode:exit() to switch between modes may cause confusion for the users of my plugin in case they try to create ModeChanged autocmds that trigger when going between custom modes like so:

autocmd ModeChanged customMode1:customMode2 ...

Since your solution switches custom modes via normal mode they would have to listen for n:customMode2, but then there's no way to differentiate between switching libmodal modes and entering a libmodal mode from normal mode.

One alternative solution that might be a little cleaner API-wise could be to add a new libmodal.mode.switch() function which switches to another libmodal mode without stacking. I still think Mode:exit() is very useful to have as well for other use cases.

What do you say? Would that be possible to implement? 🙂

[Iron-E]

I'm sure something like what you describe could be implemented— I just want to make sure I understand the details first.

Since your solution switches custom modes via normal mode they would have to listen for n:customMode2, but then there's no way to differentiate between switching libmodal modes and entering a libmodal mode from normal mode.

tl;dr: if it's exiting to normal before entering the new mode, that's a bug, and if it's happening to you I'll fix it 😁

Just to be clear, self:exit() doesn't immediately return to Normal mode. It works like this:

0. A Mode is essentially a loop over vim.fn.getchar.
1. getchar receives the pressed character, which is (in this case) mapped to a function that calls self:exit().
2. self:exit() flags a desire to exit next loop iteration.
3. Before getchar is called again, the exit flag is checked, so the mode terminates.

So, to annotate the example above:

{ -- Foo mode
  n = function(self)
    -- enter Bar mode
    libmodal.mode.enter('Bar', keymap)
    -- Bar mode has exited

    -- flag Foo mode to exit instead of taking input again
    self:exit()
  end,
}

Because of this, self:exit() can even come before libmodal.mode.enter(…) and the result will (if it works correctly) be the same.

Thus ModeChanged n:Bar and ModeChanged Foo:Bar will (again, ideally) be different.

One alternative solution that might be a little cleaner API-wise could be to add a new libmodal.mode.switch() function which switches to another libmodal mode without stacking.

Which of these suits your needs best?

-- 1. Swap means "enter another mode and after it exits, exit the current mode"
libmodal.mode.swap = function(...)
  return function(self)
    libmodal.mode.enter(...)
    self:exit()
  end
end

-- 2. Swap means "exit the current mode, and then enter a new mode"
libmodal.mode.swap = function(...)
  return function(self)
    self:exit(function() -- ← this doesn't work right now, just bikeshedding!
      libmodal.mode.enter(...)
    end)
  end
end

[mawkler]

Ah, you're right!

Which of these suits your needs best?

I'm not sure what the difference would be between the two, but to me the latter one makes more sense semantically since it first exit, then enter the next mode, right? Would there be any difference in behaviour between the two versions?

[Iron-E]

Would there be any difference in behaviour between the two versions?

Ultimately the big difference will be in the ModeChanged events. For example:

{ -- Foo mode
  -- `swap` v1
  n = function(self)
    libmodal.mode.enter('Bar', {})
    self:exit()
  end,

  -- `swap` v2
  m = function(self)
    self:exit(function() -- ← again, this is hypothetical
      libmodal.mode.enter('Bar', {})
    end)
  end,
}

Entering this mode (from Normal) emits ModeChanged n:Foo. Then:

• n pressed
  1. ModeChanged Foo:Bar (enter Bar mode)
  2. ModeChanged Bar:Foo (when Bar mode exits)
  3. ModeChanged Foo:n (exit Foo mode)
• m pressed
  1. ModeChanged Foo:n (exit Foo mode)
  2. ModeChanged n:Bar (enter Bar mode)
  3. ModeChanged Bar:n (when Bar mode exits)

[mawkler]

Hmm, I think both version should work for me so v1 is probably fine. What do you think? 🙂

[Iron-E]

I'll implement version one, it seems the simplest 👍