camlp4.4.07+1 might need to be rebuilt for ocaml 4.07.1

[Blaisorblade]

TLDR; Building OCaml software depending on camlp4 fails, but succeeds after rebuilding camlp4 from source. The failure says that camlp4 is built with an old OCaml version, and it was built before OCaml 4.07.1 was released. After rebuilding, the file-format version of camlp4 binaries is higher, consistent with the theory that camlp4 was built with an older compiler. Last, camlp4's formula, including the bottle hashes is 5 months older than ocaml's formula and bottle hashes.

So camlp4's bottle might need to be rebuilt, quite possibly after a version bump.

• have a problem with brew install (or upgrade, reinstall) a single, Homebrew/homebrew-core formula (not cask) on macOS? If it's a general brew problem please file this issue at Homebrew/brew: https://github.com/Homebrew/brew/issues/new/choose.
• ran brew update and can still reproduce the problem?
• ran brew doctor, fixed all issues and can still reproduce the problem?
• ran brew gist-logs <formula> (where <formula> is the name of the formula that failed) and included the output link?
  The build that failed was not from a formula, but here are logs for camlp4: https://gist.github.com/Blaisorblade/4a288d27d133761fd5e47de0284d6ad6.
• if brew gist-logs didn't work: ran brew config and brew doctor and included their output with your issue?

What I did.

• I installed ocaml and opam via brew; then, opam install beluga told me to install camlp4 via the system package manager, since I had installed ocaml that way, so I installed camlp4 via brew. But then building beluga failed when building ulex (a common OCaml package), and it should have succeeded instead. Rebuilding camlp4 fixed the error.

What happened:

• opam install beluga failed.

What should have happened:

• opam install beluga succeeds

Step-by-step reproduction instructions (by running brew install commands):

• brew install ocaml opam camlp4
• opam install beluga fails; (installing ulex alone should have the same problem).
• brew reinstall -s camlp4
• opam install beluga succeeds.

Full command output:

$ brew install camlp4
==> Downloading https://homebrew.bintray.com/bottles/camlp4-4.07+1.mojave.bottle.tar.gz
Already downloaded: /Users/pgiarrusso/Library/Caches/Homebrew/downloads/303b5ad8f2fb736a32a84e83a654fd79fb2e2b126148e3b024bc91dc5cfd9204--camlp4-4.07+1.mojave.bottle.tar.gz
==> Pouring camlp4-4.07+1.mojave.bottle.tar.gz
🍺  /usr/local/Cellar/camlp4/4.07+1: 165 files, 153.0MB

$ opam install beluga
The following actions will be performed:
  ∗ install camlp4 4.07+system [required by ulex]
  ∗ install ulex   1.2         [required by beluga]
  ∗ install beluga 0.8.2
===== ∗ 3 =====
Do you want to continue? [Y/n] y

<><> Gathering sources ><><><><><><><><><><><><><><><><><><><><><><><><><><>  🐫
[beluga.0.8.2] found in cache
[camlp4.4.07+system] found in cache
[ulex.1.2] found in cache

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><>  🐫
∗ installed camlp4.4.07+system
[ERROR] The compilation of ulex failed at "/Users/pgiarrusso/.opam/opam-init/hooks/sandbox.sh build make".

#=== ERROR while compiling ulex.1.2 ===========================================#
# context     2.0.4 | macos/x86_64 | ocaml-system.4.07.1 | https://opam.ocaml.org#45b0aff7
# path        ~/.opam/coq-8.8.2/.opam-switch/build/ulex.1.2
# command     ~/.opam/opam-init/hooks/sandbox.sh build make
# exit-code   2
# env-file    ~/.opam/log/ulex-61166-e4db2f.env
# output-file ~/.opam/log/ulex-61166-e4db2f.out
### output ###
# [...]
# ocamldep.opt -modules cset.ml > cset.ml.depends
# ocamldep.opt -modules ulex.mli > ulex.mli.depends
# ocamlc.opt -c -o cset.cmo cset.ml
# ocamlc.opt -c -o ulex.cmi ulex.mli
# ocamlc.opt -c -I /usr/local/lib/ocaml/camlp4 -pp camlp4orf -o pa_ulex.cmo pa_ulex.ml
# + ocamlc.opt -c -I /usr/local/lib/ocaml/camlp4 -pp camlp4orf -o pa_ulex.cmo pa_ulex.ml
# File "pa_ulex.ml", line 1:
# Error: /usr/local/lib/ocaml/camlp4/camlp4.cmi
# is not a compiled interface for this version of OCaml.
# It seems to be for an older version of OCaml.
# Command exited with code 2.
# make: *** [all] Error 10



<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><>  🐫
┌─ The following actions failed
│ λ build ulex 1.2
└─
┌─ The following changes have been performed (the rest was aborted)
│ ∗ install camlp4 4.07+system
└─

The former state can be restored with:
    opam switch import "/Users/pgiarrusso/.opam/coq-8.8.2/.opam-switch/backup/state-20190429035133.export"
$ file /usr/local/lib/ocaml/camlp4/Camlp4.cmi
/usr/local/lib/ocaml/camlp4/Camlp4.cmi:         OCaml interface file (.cmi) (Version 023)
$ brew reinstall -s camlp4
==> Reinstalling camlp4
==> Downloading https://github.com/ocaml/camlp4/archive/4.07+1.tar.gz
Already downloaded: /Users/pgiarrusso/Library/Caches/Homebrew/downloads/6a26245490ba37ae64b4c75801c33feda28e85446944aeb73cedecd47c3a6dbb--camlp4-4.07-1.tar.gz
==> ./configure --bindir=/usr/local/Cellar/camlp4/4.07+1/bin --libdir=/usr/local/lib/ocaml --pkgdir=/usr/local/lib/ocaml/camlp4
==> make all
==> make install LIBDIR=/usr/local/Cellar/camlp4/4.07+1/lib/ocaml PKGDIR=/usr/local/Cellar/camlp4/4.07+1/lib/lib/ocaml/camlp4
$ file /usr/local/lib/ocaml/camlp4/Camlp4.cmi
/usr/local/lib/ocaml/camlp4/Camlp4.cmi:         OCaml interface file (.cmi) (Version 024)
$ opam install beluga
The following actions will be performed:
  ∗ install ulex   1.2   [required by beluga]
  ∗ install beluga 0.8.2
===== ∗ 2 =====
Do you want to continue? [Y/n] y

<><> Gathering sources ><><><><><><><><><><><><><><><><><><><><><><><><><><>  🐫
[beluga.0.8.2] found in cache
[ulex.1.2] found in cache

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><>  🐫
∗ installed ulex.1.2
∗ installed beluga.0.8.2
Done.

[fxcoudert]

We can rebuild, but what is worrying is that the camlp4 test did not fail when ocaml was updated. Can you suggest an improvement to our camlp4 test block that would catch such issues?

[Blaisorblade]

@fxcoudert I program in Coq, not really in OCaml, so I'm not sure, but building ulex is at least relatively fast, if you can rely on opam for those tests. But I fear you can't?
In general, "compile and link some program with the new compiler that uses camlp4" should do it, so one could probably minimize "building ulex" to a smaller test that still fails; it's just a matter of effort. I could try, but I'm not sure I have time soon enough.

Instead, current tests just run the camlp4 binary, which is reasonable but cannot catch problems linking new binaries.

[SMillerDev]

If you could make a pull request that adds "building ulex" as a test maybe we can work from there?

[Blaisorblade]

@SMillerDev Thanks. I realized that using opam is in fact a pain (it needs to run an interactive setup wizard, install files, etc.), so I have minimized the testcase and added a test. PR coming.

[fxcoudert]

Now fixed, thanks again @Blaisorblade