How to use PackChanged with ~/.config/nvim/plugin/ file structure

[drowning-cat]

Related:

• Goals for v2 saghen/blink.cmp#1059 (comment)

Let's define a configuration as:

-- init.lua

-- plugin/completion.lua

vim.api.nvim_create_autocmd("PackChanged", {
  pattern = "blink.cmp",
  desc = "Run `:BlinkCmp build` after pack changed",
  group = vim.api.nvim_create_augroup("blink_update", { clear = true }),
  callback = function(e)
    local kind, name = e.data.kind, e.data.spec.name
    if kind == "install" or kind == "update" then
      vim.cmd.packadd({ args = { name }, bang = false })
      require("blink.cmp.fuzzy.build").build()
    end
  end,
})

vim.pack.add({
  { src = "https://github.com/Saghen/blink.cmp", version = "main" },
})

require("blink.cmp").setup()

I would expect the PackChanged event to be fired after installation of blink.cmp, since it is defined before vim.pack.add. However, a problem arises when the nvim-pack-lock.json file is present. In this case, Neovim installs plugin specs before loading files from plugin/, which causes the PackChanged to be ignored for the install event.

This behavior feels unintuitive and effectively forces to define PackChanged autocmd in init.lua to cover both install and update cases. That's not ideal, since the related logic ends up being split across different places instead of living in one.

[SebasF1349]

Looks to be related to #35314 and the linked pr.

[shushtain]

According to :h startup, aren't your <config>/plugin/... (if I understood your naming correctly) compete with step 10 (loading the actual local and external plugins)?

You don't need to keep actual lines in init.lua, but you could still put these pack.add()s and their PackChanged into <config>/lua/plugin/foo.lua and require("plugin.foo") from your init.lua, so they are loaded at step 7 (loading config files). You know, like we did with Lazy essentially...

plugin folder is for quite independent to your setup plugins, like adding some additional user commands, etc. Those files are executed way after your "config files" are processed, which is too late based on the current limitations of vim.pack.add().

[shushtain]

To be clear, lua/plugin/ could just as well be named lua/banana/. These are not special folders, just ways to structure your config.

[drowning-cat]

I am trying to use the MiniMax approach, but with vim.pack instead of mini.deps, where almost all files are located in the <config>/plugin/, in a fashion similar to pkazmier's configuration files.

For reference, MiniMax has the following folder structure:

.
├── after/
├── plugin/
│   ├── 10_options.lua
│   ├── 20_keymaps.lua
│   ├── 30_mini.lua
│   └── 40_plugins.lua
├── snippets/
└── init.lua

Since vim.pack was inspired by mini.deps, I thought MiniMax behavior should be relatively easy to script with such file structure.

[shushtain]

Well, you do you. I have no idea what mini.deps does under the hood but it doesn't mean the structure it uses is guaranteed to work without the plugin manager itself. I mentioned :h startup to explain why that seems counterintuitive. Your initial post seemed to imply there are two options: cluttered init.lua or putting config into plugin/ — which is neither typical nor an existing duality.

Manual bookkeeping through nn_ does not seem to be any less tedious than requiring modules from init.lua according to the same specific flow. You wouldn't want to load a plugin that relies on tree-sitter or vim.g.mapleader before they are initialized, so there seems to be no advantage to this "just read from plugin/" approach.

[drowning-cat]

I was a little bit off with my assumptions. Essentially, all PackChanged events that use kind=install need to be defined before the first use of vim.pack.add because the presence of nvim-pack-lock.json changes when the install event is triggered for all defined patterns; it bubbles up to the first vim.pack.add.

Instead of:

-- /foo.lua
autocmd kind=install foo
packadd foo

-- /bar.lua
autocmd kind=install bar
packadd bar

We need to bubble the autocmds:

-- /init.lua
autocmd kind=install foo
autocmd kind=install bar

-- /foo.lua
packadd foo

-- /bar.lua
packadd bar

As mentioned before, this behavior can be unintuitive, but I don't know if it should be fixed in the first place, so I'm closing this discussion.

[folliehiyuki]

I was also caught by this issue when migrating my Neovim config to 0.12. The event is indeed fired for all plugins present in the lockfile (rather than the one matching the defined pattern) after the first vim.pack.add(), due to lock_read() -> lock_sync() -> install_list() call.

Is this intentional, @echasnovski ? Sorry for the ping! I just want to make sure I don't do anything unintentionally stupid in my own config.

[echasnovski]

Probably yes, because:

• PackChanged[Pre] are triggered whenever a plugin is changed on disk.
• On first install, "... all plugins from the lockfile will be installed at once and at lockfile's revision (instead of inferring from version). ...". Meaning that installation of each plugin will trigger PackChange[Pre] with kind=install.
• There is also a note in the example of hooks.

It is a "probably yes", since I don't really understand "(rather than the one matching the defined pattern)". Each PackChanged[Pre] is triggered separately for every plugin while supplying full path as pattern, so I don't think autocommand pattern should have problems here.

[drowning-cat]

I think the main point of confusion for many users with vim.pack is the requirement to define all PackChanged autocmds before the first vim.pack call.

What I want is to define autocmds related to a specific vim.pack.add call right next to it, without splitting the configuration across different places, which makes cleanup inconvenient. Right now, we have to worry about whether a lock file will be used or where the first vim.pack.add call is made, whether it's in a plugin, in init.lua, or somewhere else. In practice, users expect install (update?) events to be triggered when vim.pack.add is loaded.

Maybe a separate "pack lock" event could help, or some kind of store that tracks plugin status, but those solutions feel at least clumsy.

Personally, I tried something like this:

Code

local M = {}

setmetatable(M, { __index = vim.pack })

function M.now(callback)
  callback()
end

function M.later(callback)
  vim.schedule(callback)
end

local is_build_event = function(e)
  return e.data.kind == "install" or e.data.kind == "update"
end

local pending_events = {}

local pull_pending = function(src)
  local matches, remaining = {}, {}
  for _, e in ipairs(pending_events) do
    if e.data.spec.src == src and is_build_event(e) then
      table.insert(matches, e)
    else
      table.insert(remaining, e)
    end
  end
  pending_events = remaining
  return matches, remaining
end

vim.api.nvim_create_autocmd("PackChanged", {
  group = vim.api.nvim_create_augroup("pack_build", { clear = true }),
  callback = function(e)
    local build = e.data.build
    if e.data.active == false then
      table.insert(pending_events, e)
    elseif build and is_build_event(e) then
      build(e)
    end
  end,
})

-- ```lua
-- pack.add({
--   {
--     src = "https://github.com/nvim-treesitter/nvim-treesitter",
--     data = {
--       build = function()
--         require("nvim-treesitter").update()
--       end,
--     },
--   },
-- })
-- ```
function M.add(specs, opts)
  vim.pack.add(specs, opts)
  for _, spec in ipairs(specs) do
    local build = vim.tbl_get(spec, "data", "build")
    if build then
      local pending = pull_pending(spec.src)
      if not vim.tbl_isempty(pending) then
        build(pending[1], pending)
      end
    end
  end
end

return M

This lets us use spec.data.build, but the approach on itself feels limiting. For example, the build might fail, or you might accidentally close neovim before it finishes (forked processes are probably terminated). In that case, you have to rebuild manually.

blink.cmp solves this in a much simpler way for vim.pack (see https://main.cmp.saghen.dev/installation.html#vim-pack):

vim.pack.add({ 'saghen/blink.lib', 'saghen/blink.cmp' })
local cmp = require('blink.cmp')
cmp.build():wait(60000) -- no-op call
cmp.setup()

Although waiting up to 60 seconds before entering neovim is also not the best option.

[echasnovski]

What I want is to define autocmds related to a specific vim.pack.add call right next to it, without splitting the configuration across different places, which makes cleanup inconvenient. We literally have to care about where the first call is made whether it's in a plugin, in init.lua, or somewhere else. In practice, users expect install events to be triggered when vim.pack.add is called.

I did think about several possible solutions to this and all of them provided trade-offs that I don't think are worthwhile:

• All of them add very non-trivial complexity to the code, which is already kind of not that straightforward.
• Delaying installation from the lockfile until its corresponding vim.pack.add() is problematic: plugins are not installed in parallel, there needs to be a VimEnter event for installing lazy loaded plugins (with some kind of tracking), the vim.pack.get() can provide not relevant information.

My assessment of the trade-offs still stands. This problem manifests in a very rare combination of events (compared to the number of times Neovim starts up): 1) single startup; 2) when there is already a lockfile; 3) for a plugin that needs an installation hook. Introducing code+maintenance+tests complexity trade-offs in this case is not quite worth it. Documenting to have all installation hooks before the first vim.pack.add() feels acceptable to me. And in the worst case, doing something manually once per setting up Neovim config is not that bad. Not everything needs to be 100% fully automated, but it is still possible to achieve that.

The long term plan is to alleviate this problem further is with 'pkg.json' support when plugins define installation hooks themselves.

[drowning-cat]

TL;DR

Not worth it. Wait for pkg.json.