Spec out the new plugin system

[pirog]

We need to figure out exactly how we want the new plugin system to work:

1. • What commands do we want to install/remove/update plugins
2. • How do we want user configuration eg a plugins key inside of a Landofile to work?

A lynchpin question here is "can we load plugins from inside an application's repo" and have them basically work as "on demand" global plugins. @pirog will investigate this question and its answer will likely influence which way we go.

---

Alright so the answer to "can we load plugins from inside an application's repo" is "yes" although it will require some changes to Landos code. With that in mind, here is my proposal/first pass on the plugin stuff

1. Changes to Lando core

• Be able to identify very early eg before Lando bootstraps that an application has plugins in its codebase and to add those to config.pluginDirs so they are loaded correctly
• Do not require that plugins exist in the plugins directory for each pluginDir specified
• We may also need a mechanism to "re-bootstrap" lando for the use case of "plugins get installed" as part of a build step, this way the plugins can get installed first and are then available immediately.

2. Docs as spec

Here is what the docs for the feature might look like

CLI Usage

Global

# Add/remove a plugin globally
lando plugin:add -g myplugin
lando plugin:remove -g myplugin

# The "package" convention should be the same as the YARN spec but should also 
# resolve "official lando packages"
lando plugin:add -g lando-power@1.2 # behind the scenes this might actually do `yarn add lando/lando-power#1.2`
lando plugin:add -g pirog/piroplugin#commitref
lando plugin:add -g my-plugin-on-npm@^1

Behind the scenes these plugins will get installed using a node:12 lando service in the /lando/plugins directory which would put them in ~/.lando/plugins on your host, a yarn install will also be run if the plugin has package.json. The installation mechanism will basically be mapping lando plugin:add -g lando-power@1.2 to yarn global add lando-power@1.2 --global-folder=/lando --modules-folder=plugins inside the container. This should allow us to inherit a lot of yarns awesome stuff without needing to reinvent a whole plugin management system.

Removal can happen host side.

Application

The above commands can also be run from within a Lando app context without the -g flag

# Add/remove a plugin globally
lando plugin:add myplugin
lando plugin:remove myplugin

# The "package" convention should be the same as the YARN spec but should also 
# resolve "official lando packages"
lando plugin:add lando-power@1.2 # behind the scenes this might actually do `yarn add lando/lando-power#1.2`
lando plugin:add pirog/piroplugin#commitref
lando plugin:add my-plugin-on-npm@^1

This will work similarly to the global installation as far as using a node:12 service to yarn add/install things in the correct place. However in this situation the modules will be installed in a lando_plugins folder in the root of the app repo. The intention is for this folder to be gitignored a la node_modules or vendor.

Additionally, the plugin will be added to the Landofile using a YAML format version of package.json isms.

name: myapp
plugins:
  myplugin: "1.2"
  lando-power: "^1

Configuration

In addition to the global config options already available for Lando plugins you can configure your plugin dependencies in one of two ways in your Landofile. The method above and an array of directories relative to your app root that may contain plugins.

name: myapp
plugins:
  myplugin: "1.2"
  lando-power: "^1"
pluginDirs:
  - plugins

plugins should be used to install third party plugins or for your own external plugins that you use on more than one app. pluginDirs allows you to load plugins that are defined directly in your code repo. This is good for apps that require special "one-off" applications.

plugins should be installed VERY early on when you lando start then lando should rebootstrap if needed so they are available for the rest of the lando start. A good example of this is lando start on a repo that uses a recipe provided by a third party plugin. This should be using the same underlying install mechanism as the CLI lando plugin:add command.

pluginDirs should be loaded right after plugins and should have the highest priority eg if Lando finds two of the same plugin in plugins and pluginDirs it uses the one in the latter.

3. Implementation strategy

There are basically four chunks of work needed to get this all rolling.

1. Lando core changes
2. Loading plugins directly from an applications codebase
3. CLI commands to install/remove apps
4. Automatically installing apps listed in a Landofile

Lando core changes is the obvious first thing we need to do. @pirog might be best suited to getting that wrapped up. After that i think we do 2, 3, 4. The thinking here is

1. Its probably pretty easy to get 2 rolling and that means people can immediately begin to distribute plugins.
2. 3 is more involved and requires we correctly configure the underlying mechanisms for lando to install and remove plugins generally
3. 4 basically ties 2 and 3 together.

@dustinleblanc @serundeputy let me know what you think. Writing it all out_feels_ like less work than i had thought. Interested if this implementation would satisfy your visions of the system and if not id love feedback.

Update:

Here are the big three tickets left that we need to do. note that many of the core updates have been done:

1. Create global plugin mgtm commands #2493
2. Create commands for app level plugin management #2502
3. Auto install app plugins on lando start/rebuild #2503

[pirog]

@dustinleblanc @serundeputy i did most of 1 and 2 as part of our website work so that should at least give us some minimal ability to ship plugins in 3.0.x for now.
https://github.com/lando/lando/pull/2466/files

Note that the PR has some other work in it, but the changes for app loading plugin support was very straightforward.
https://docs.lando.dev/config/plugins.html

[pirog]

I've spin up the tickets we need to get this rolling and put them in the issue description so going to close this out.