Fix generation of -pp in .merlin

[rgrinberg]

First of all, we are going to be more strict and error out if we can't expand
any variables for the .merlin. If this happens, then it's really a bug that we
should catch.

The functions that swallow expansion errors are removed because they're too easy
to misuse.

Signed-off-by: Rudi Grinberg rudi.grinberg@gmail.com

[rgrinberg]

A real fix for this ends up being far more complicated than I expected. However, there's still the case where the preprocessor is defined in the same directory as the libraries it is preprocessing. For example:

(executable
 (name main)
 (modules main))

(library
 (name foo)
 (preprocess (action (run ./main.exe %{input-file})))
 (modules foo))

Fundamentally, there's no way to generate a correct .merlin here. My only workaround here is to just give up and turn off the preprocessing in the .merlin.

The reason why this is a problem now is that I'd like to support correct preprocessing specifications such as those in omp:

(library
 (name migrate_parsetree)
 (public_name ocaml-migrate-parsetree)
 (wrapped
  (transition "Access modules via the Migrate_parsetree toplevel module"))
 (libraries compiler-libs.common result ppx_derivers)
 (modules :standard \ migrate_parsetree_driver_main)
 (preprocess (action (run %{exe:../tools/pp.exe} %{read:ast-version}
                       %{input-file})))
 (ppx.driver
  (main       Migrate_parsetree.Driver.run_main)
  (flags      --dump-ast)
  (lint_flags --null)))

This requires us to build ast-version which means that we now need to discover and add the dependencies from the preprocessing action.

[ghost]

The case you described is now handled by this PR right?

[rgrinberg]

The case you described is now handled by this PR right?

Before my PR:

• Handling of (preprocess (action ..)) would be mostly broken. It would only work in cases where only static variables where used in the preprocessing specification.

• Directories with mixed preprocessing specifications such as no_preprocessing, and (preprocess (action ..)) would approximate with (preprocess (action ..)). This isn't always sound as the preprocessor can be defined in the same dir as the sources it's preprocessing.

After my PR:

• Handling of (preprocess (action ..)) is always correct when generating .merlin.

• Directories with mixed preprocessing specifications such as no_preprocessing, and (preprocess (action ..)) will no longer approximate and use no_preprocessing. This is technically more correct, but is a bit less useful. However I think that having correct a correct .merlin whenever it's possible is more important than supporting use cases that .merlin doesn't handle well today.

[rgrinberg]

I rebased the PR and fixed the spurious changes in the other tests.

[ghost]

Oh, I see. Indeed I agree that these cases are dodgy.