[Feature proposal] Add more possible commands in build_systems

[Lattay]

Hello,
In order to implement properly #1 I need the Build command to do be able to use different command templates depending on its parameters.
For example if I add the following to the s:build_systems dict:

  \   'Dune':
  \   {
  \     'file'    : 'dune',
  \     'command' : 'dune',
  \   },

Basically nothing work as expected:

• Build command produce the dune help message
• Build run initialize the build directory but do nothing more
• Build build actually build the project
• Build exec ./%:r.exe would actually run the current file if it is defined in a dune file as an executable to be buit and the only dune file is in the current working directory of vim.

So for a retro compatible fix of this I would propose that the 'command' item of a build system could be either a string, in which case nothing changes, or a dictionary of specific commands.
This dictionary would be similar to s:language_cmds items:

• keys would correspond to the first argument from the Build command, and 'build' would be used by default
• the commands themselves would be templates (using s:prepare_cmd_for_shell) and ran from the location of the first build system file found

Also, a new magic should be added to s:prepare_cmd_for_shell: %RELPATH% being the relative path directory where the command will be ran to the current file.

This way I could implement my previous example as:

  \   'Dune':
  \   {
  \     'file'    : 'dune',
  \     'command' : {
  \       'build' : 'dune build',
  \       'clean' : 'dune clean',
  \       'run'   : 'dune exec %RELPATH%/%HEAD%.exe',
 \      },
  \   },

Eventually, a 'base' key could be added to the 'command' dict (in this case the value would be 'dune') which would allow handling unknown arguments to Build (:Build runtest would expand to dune runtest using 'base').

I would happily implement all of this but I do not want to take over your project @AlxHnr so what do you think ?
Does this conform to what you want for this plugin ?

[Lattay]

I made the implementation and application in my branch add_dune if you want to see it.
If you are OK with this I may update the doc and create a PR just for that or use #1 depending of what you prefer.

[AlxHnr]

TLDR: All build systems suck and I couldn't hide them behind a consistent interface. If you have a solution to the problems I've ran into (see below), I will listen.

I just removed an almost identical feature a while ago, see c0f0c26. But unlike your proposal I used the content of command for what you describe as the base key. So instead of your example you could do:

-   \       'build' : 'dune build',
-   \       'clean' : 'dune clean',
-   \       'run'   : 'dune exec %RELPATH%/%HEAD%.exe',
+   \       'build' : 'build',
+   \       'clean' : 'clean',
+   \       'run'   : 'exec %RELPATH%/%HEAD%.exe',

It had a few problems tough. If run is always replaced with exec ..., but I want to skip this replacement just once, how do I do it? Switch to a terminal and run dune run manually? Or add some non-obvious flag like :Build! or :Build --no-replace? Two commands to achieve almost the same thing would require users to either have two mappings for everything, or write a custom wrapper which decides what to do depending on the situation. And that would kinda defeat the point of this plugin.

• Build command produce the dune help message

I agree, the idea of this plugin is just to be able to run :Build and expect it to work. It could be fixed by falling back to the build target by default. One would just need to define this target:

  " somewhere in the config:
  \    "build": "my-maker --fast --build-all"

Now :Build and :Build build will do the same. You just need to set the build target to an empty string for build systems which work without any arguments:

  " E.g. GNU Make:
  \    "build": ""

Both :Build and :Build build will now work and simply call make without any arguments. But what if your Makefile has a target named build? This would require you to use the following command:

:Build build build

Not only that, but depending on the replacement strings the following commands may do completely different things:

:Build clean
:Build build clean

:Build build --custom=true
:Build --custom=true

This is confusing to new users without making it obvious why. This particular issue may be solved by always requiring to specify the build target. E.g. just plain :Build with no arguments will display a help message telling you to use :Build [TARGET].

But I'm not sure if thats the right way to go. Every argument passed to :Build ... is highly dependent on the underlying build system. Trying to hide this works only for very basic tasks. Everything beyond that will require to either add more replacement magic to this plugin, or add more wrapper code to my own vim config. And since I already have enough magic in my own config, I decided to remove this feature.

Now I can at least guarantee that :Build foo behaves identically to make foo, mvn foo or dune foo. But its impossible to guarantee that :Build foo behaves the same across all build systems. The language-specific fallback commands can be seen as its own build system, so its consistent.

• Build exec ./%:r.exe would actually run the current file if it is defined in a dune file as an executable to be buit and the only dune file is in the current working directory of vim.

What if you want to run a different file from the file you are editing? How does it fit your workflow?

[Lattay]

So I started by answering your points one by one but a general point of view emerged from my answers.
I will explain a very oriented opinion that may not fit the philosophy you want for your plugin.
Again, this is your project, so I can understand that you don't like my proposal.
Also, I think it actually fits what you want to achieve if I understood it well.

Here is how I think this plugin could be used:
There is only two valid calls to the :Build command:

• :Build without any arguments would call the default command for building, thus be equivalent to :Build build
• :Build [subcommand] ... would call the corresponding command with eventual arguments

Subcommands should not be specific to the underlying build system.
Whether Build foo corresponds to mvn foo or mvn bar while it correspond to make fizz or make baz should be irrelevant as far as Build foo have the same overall effect for both mvn and make.
Build buildshould always build the current project, Build run should always run the current file, Build clean should always clean the current project.

Arguments to subcommands should be specific to the underlying build system and should be used by the user for more advanced tweaking. The plugin should not take care of them and only pass them to the underlying build system.

---

Given that framework to answers your individual points:

It had a few problems tough. If run is always replaced with exec ..., but I want to skip this replacement just once, how do I do it? Switch to a terminal and run dune run manually? Or add some non-obvious flag like :Build! or :Build --no-replace? Two commands to achieve almost the same thing would require users to either have two mappings for everything, or write a custom wrapper which decides what to do depending on the situation. And that would kinda defeat the point of this plugin.

If you want to call dune run ... you would use Build do run ... because dune run does not have the same semantic as Build run: dune run cmd stands for "execute cmd in an environment where the current project have been install as a package".
Advanced use cases imply advanced commands, that is only fair.

Both :Build and :Build build will now work and simply call make without any arguments. But what if your Makefile has a target named build? This would require you to use the following command:

:Build build build

Not only that, but depending on the replacement strings the following commands may do completely different things:

:Build clean
:Build build clean

:Build build --custom=true
:Build --custom=true

:Build build can only mean "call the subcommand build without arguments" and :Build build build would be perfectly valid and semantically pretty clear. It is not too pretty but since it is project specific you cannot expect to abstract it away.
In fact, I do use a build target in my makefiles but I never use it manually because it only do mkdir build.

:Build clean and :Build build clean are semantically very different: "call the command for clean" and "call the command for build with the argument clean" respectively, so it is normal that they have different behavior.
I cannot even think to a single build system that is not make based and would still end up doing the same thing for those two commands.

:Build build --custom=true would do just what you expect while :Build --custom=true would simply be invalid because of the lack of a known subcommand.

But I'm not sure if thats the right way to go. Every argument passed to :Build ... is highly dependent on the underlying build system. Trying to hide this works only for very basic tasks.

In my proposal, the first argument passed to :Build is not dependent on the underlying build system.
Anything else is dependent on the underlying build system and should be totally left to the end user.

Now I can at least guarantee that :Build foo behaves identically to make foo, mvn foo or dune foo. But its impossible to guarantee that :Build foo behaves the same across all build systems. The language-specific fallback commands can be seen as its own build system, so its consistent.

As I said earlier, whether :Build foo translate to make foo or make barshould not be considered globally, but only in the 'Make' build system specifically.
What should be ensured is that :Build foo behaves the same across all build systems.
My proposal allows that by providing independent commands for each subcommands.

• Build exec ./%:r.exe would actually run the current file if it is defined in a dune file as an executable to be buit and the only dune file is in the current working directory of vim.

What if you want to run a different file from the file you are editing? How does it fit your workflow?

I don't want to use Build exec ./%:r.exe anyway, I was just looking for workarounds. This command is too complex in my opinion to be a valid use case for this plugin. If it was the only solution to run a file with the plugin I would rather open a terminal and type the command by hand.

As for actually running a different file, my proposal provides two solutions:
Either open the file and :Build run (I would definitely do that in my normal workflow), or run :Build do exec path/to/other/file.exe.
The latter sucks but I don't see a reason why I would want to specifically run a different file with your plugin instead of simply firing a :terminal.

I think this plugin is good because its scope is small: abstract away common tasks behind an homogeneous interface.
Anything too advanced should not be included as it get out of the scope and could be done simply with other solutions (:terminal and :sp term://cmd in particular).
However, the configuration interface should be flexible enough to allow any build system to be used identically.
My proposal try to answer those two requirements without changing your code too much.

[AlxHnr]

Your views on what a sub-command should be make a lot of sense. The do command seems to be the key here. I don't know why I didn't come up with this, but it seems really obvious now.

Either open the file and :Build run (I would definitely do that in my normal workflow), or run :Build do exec path/to/other/file.exe.

I can see the need for having a run command which depends on the current buffers name. I don't know if it would interfere with anything, but it would definitely open a lot of doors to other build systems where you want to run the tests from the current file only.

I will be happy to merge such a feature if you implement it. These things are important to me:

• Allow users to extend/override sub-commands via g:build#systems
• Document sub-commands in help page, README and code examples
• Document %RELPATH%
• Add available sub-commands to help message which is shown when user enters wrong command. I already do this for language-specific fallback commands and it is pretty useful.

The function build#target() takes only one string as argument, which means you have to split the sub-command manually. This is intentional to prevent Vim from splitting arguments into <f-args>. Calling :Build --flag="-Wall -Bar ' \ with five white-spaces should pass the exact string to the build system, including all five white-spaces, slashes and quotes.

[Lattay]

Very happy to see that it fits you.

I will be happy to merge such a feature if you implement it. These things are important to me:

• Allow users to extend/override sub-commands via g:build#systems
• Document sub-commands in help page, README and code examples
• Document %RELPATH%
• Add available sub-commands to help message which is shown when user enters wrong command. I already do this for language-specific fallback commands and it is pretty useful.

The function build#target() takes only one string as argument, which means you have to split the sub-command manually. This is intentional to prevent Vim from splitting arguments into <f-args>. Calling :Build --flag="-Wall -Bar ' \ with five white-spaces should pass the exact string to the build system, including all five white-spaces, slashes and quotes.

I already implemented a proof of concept but it does not fit all those requirements, I will work on this.

[AlxHnr]

:BuildInit could be replaced with an init subcommand. This would simplify the interface.

I'm not sure the same could be said for :BuildInfo. But its an interesting idea to allow a new type of internal, fixed subcommands:

• :Build info
• :Build help
• :Build lchdir

[Lattay]

Would you like to migrate all available build systems to this new configuration mode (while still keeping the old one available) ?
That would provide more homogeneous help messages and a common subcommand set.
Also the documentation would be more relevant if the subcommands are used for all available build system as opposed to only for dune.

EDIT: Actually as I look at writing help for the new feature I realize how hard it is to explain both systems together because the new one reflect the behavior of the plugin when there is no build system, while the old one reflect the semantic of make (the concept of subcommand versus the concept of target)

[Lattay]

The more I read the sources and the doc, the more I realize how similar are what I added and what you did for fallbacks and I think you really did a good job for that.
I think it would be nice if the interface was exactly the same with a build system.

[AlxHnr]

On 10/4/20 10:45 PM, Théo Cavignac wrote:

Would you like to migrate all available build systems to this new
configuration mode (while still keeping the old one available) ?
That would provide more homogeneous help messages and command set.

I don't know what you mean with keeping "the old one available". There should be only one way to declare build systems. If you have to move the old ones to this new system, then do it. Just don't forget to do the same with g:build#systems and its documentation. Breaking backwards compatibility is ok. This plugin has only 5 users so it won't be that much of a problem. If you want to move :BuildInit into a subcommand, it should display a deprecation warning like Please use ":Build init" instead. I will remove the deprecation warning in a few months from now together with the command.

I think it would be nice if the interface was exactly the same with a
build system.

Yes, it will probably end up like this. Fallback commands won't have a do subcommand tough, but thats ok imo.

[AlxHnr]

EDIT: Actually as I look at writing help for the new feature I realize how hard it is to explain both systems together because the new one reflect the behavior of the plugin when there is no build system, while the old one reflect the semantic of make (the concept of subcommand versus the concept of target)

This should be solvable by removing the "old" system. The subcommand way of doing it is consistent, superior and should be the only way to do it.

[AlxHnr]

Fixed on master.