[mini.snippets] "MiniSnippets.parse" performance improvement.

[Aumnescio]

Bit lazy about posting a full issue / PR about this, but want to share. This primarily concerns parsing large'ish snippets. (like 100-200+ lines.) Though the performance impact should affect basically all parsing of snippets. It's just not really so noticeable unless the snippet is a bit larger.

This part of snippets.lua is seemingly very slow. (~375ms to parse the attached sample file snippet.)

  for i = 0, vim.fn.strchars(snippet_body) - 1 do
      local depth = #state.depth_arrays
      local arr = state.depth_arrays[depth]
      local processor, node = H.parse_processors[state.name], arr[#arr]
      processor(vim.fn.strcharpart(snippet_body, i, 1), state, node)
  end

Alternative version

The code below parses the same snippet in under 5ms. It also passes all the snippets tests.

  local i, n = 1, #snippet_body

  while i <= n do
      local byte = snippet_body:byte(i)
      local character

      if byte < 0x80 then
          -- ASCII
          character = string.char(byte)
          i = i + 1
      else
          -- Consume one UTF-8 codepoint as opaque text.
          local len =
              byte < 0xE0 and 2 or
              byte < 0xF0 and 3 or
              4

          character = snippet_body:sub(i, i + len - 1)
          i = i + len
      end

      local depth = #state.depth_arrays
      local arr = state.depth_arrays[depth]
      local processor, node = H.parse_processors[state.name], arr[#arr]
      processor(character, state, node)
  end

Usage: just replace the original loop block with this version.

Attachments

NOTE: Github doesn't allow me to upload the file ending in .lua, so it needs to be renamed. (for example to toml.lua)

sample-toml-lua.txt

[echasnovski]

Yeah, I can reproduce the performance difference with and without using vim.fn.strchars + vim.fn.strcharpart.

I think it would be safe to just use byte iteration, but need more time to confer and test.

However, even the existing approach is usually done together with caching file contents. I am not so keen on introducing custom character byte length logic (and existing tests might have missed something to do with multibyte characters). So if the "just traverse bytes" solution doesn't work out, I'll leave it as is.

[echasnovski]

I think using raw bytes instead of characters when traversing the snippet should be fine indeed. So this should now be how latest main does it.