Support command substitution in .merlin files

[gasche]

Again for Coq users, it would be convenient to have something like

FLG -rectypes
S $(coqc -where)/**
B $(coqc -where)/**

for their .merlin files in plugins. The reason why is that they have an ossified workflow where Coq is installed in various places and not tracked by ocamlfind (don't ask).

I think the feature could be of general interest for other Merlin users, but there would be two arguments against this:

• it might encourage the use of .merlin files that are not portable to some systems (by using non-portable shell commands)
• there are security issues with giving yet another way to execute code on user's machines

Do you think the feature request is a good idea? If so, I might try to implement it myself if you don't have time.

[gasche]

cc @pirbo with which I discussed the issue (as the evil coq-libraries-compilation plan master).

[let-def]

I am not sure relying on shell-expansion is fixing the problem at the right level (although it might be the "unixy"-way). Your suggestion to use findlib looks much better to me.
In other word, what is needed is a declarative way to specify project integration and dependencies, and findlib is rather good at it.

(In the meantime Coq users can either use elisp code to automatically do the setup when opening a new buffer, or maybe write a temporary META file while waiting for a proper one from upstream).

[dbuenzli]

I think you definitively want to keep .merlin files purely declarative. Maybe merlin should maybe able to specify a convention for defining packages in a given package-system and associate a package resolver to it.

[let-def]

Isn't generation of .merlin as a byproduct of configuration/build process a viable option? (I would love living in a world without configure and with proper declarative build systems, but that doesn't seem to be a major concern of unixes :)).

The other option I consider would be to extend .merlin files with a shebang-like mechanism.
If the .merlin starts with #!name, merlin would-look for an ocamlmerlin-$name binary and filter the file through it (or report it as missing).
That should enable @dbuenzli suggestion of custom package resolvers and to some extent your request, without opening a blatant breach of security, as the attacker would need at least a control over PATH and ability to put arbitrary executable files on the host system.
Of course, the next point of failure would be in package resolvers.

[gasche]

I have mixed feelings about the idea of "staying declarative" when that means more pain for users down the road that have to hack their needs around the tool in various ways. System command expansion is a time-honored "most flexible way to do things" that is supported by all systems (Windows included), and while all the use-cases that have been discussed here and elsewhere (accessing opam variables, accessing ocamlfind variables, asking Coq where it is installed) would work fine under all operating systems, while the complex and possibly-over-engineered workaround suggested here have gone nowhere and seem insufficient to handle some of them.

(Re. security: .merlin files are distributed with source OCaml packages, which are themselves, kind of bind design, full of code to be run on a local machine. I wouldn't worry about .merlin files being the weakest point of attack.)

[gasche]

(Note: I would also be fine with support the %{...}% syntax for opam variables, and restricting merlin path to such opam variables.)

[rgrinberg]

@gasche as far as I understand, coq users are converging are dune. The best way to support them would be to improve the integration of merlin + dune. Adding new features in .merlin is unlikely to help things because these .merlin files are almost impossible to write correctly by hand, so I'm closing this issue. If you have any objections, please re-open :)