camlp4 broken on reinstall

[mgree]

I've had some recent issues with ExtUnix on OS X. After finding some discussion, I suspect that the core issue is one of camlp4 being inconsistent after an OPAM reinstall.

The order of events for me was:

1. opam update
2. opam upgrade (ExtUnix doesn't install)
3. opam uninstall camlp4
4. opam install camlp4
5. opam install ExtUnix (successful)

I'm loathe to try to do more to repeat the issue because I can't get work done without ExtUnix and am now afraid to break things.

Perhaps it's related to #140? Looking at ExtUnix's CI logs, I suspect the issue is something specific to OCaml 4.07...

# opam config report
# opam-version      2.0.1 
# self-upgrade      no
# system            arch=x86_64 os=macos os-distribution=homebrew os-version=10.13.6
# solver            builtin-mccs+glpk
# install-criteria  -removed,-count[version-lag,request],-count[version-lag,changed],-changed
# upgrade-criteria  -removed,-count[version-lag,solution],-new
# jobs              4
# repositories      1 (http) (default repo at 393c2feb)
# pinned            1 (version)
# current-switch    4.07.1+flambda

[avsm]

Were you using Homebrew to get camlp4 in the first instance? I suspect the problem lies with how camlp4 is packaged in Homebrew. It doesn’t appear to have been updated when OCaml was updated to 4.07.1, and hence the binary package was inconsistent.

…

On 3 May 2019, at 13:21, Michael Greenberg ***@***.***> wrote: I've had some recent issues with ExtUnix on OS X. After finding some discussion, I suspect that the core issue is one of camlp4 being inconsistent after an OPAM reinstall. The order of events for me was: opam update opam upgrade (ExtUnix doesn't install) opam uninstall camlp4 opam install camlp4 opam install ExtUnix (successful) I'm loathe to try to do more to repeat the issue because I can't get work done without ExtUnix and am now afraid to break things. Perhaps it's related to #140? Looking at ExtUnix's CI logs, I suspect the issue is something specific to OCaml 4.07... # opam config report # opam-version 2.0.1 # self-upgrade no # system arch=x86_64 os=macos os-distribution=homebrew os-version=10.13.6 # solver builtin-mccs+glpk # install-criteria -removed,-count[version-lag,request],-count[version-lag,changed],-changed # upgrade-criteria -removed,-count[version-lag,solution],-new # jobs 4 # repositories 1 (http) (default repo at 393c2feb) # pinned 1 (version) # current-switch 4.07.1+flambda — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[mgree]

It seems so:

$ brew info camlp4
camlp4: stable 4.07+1 (bottled), HEAD
Tool to write extensible parsers in OCaml
https://github.com/ocaml/camlp4
/usr/local/Cellar/camlp4/4.07+1 (165 files, 153.0MB) *
  Poured from bottle on 2018-09-26 at 15:28:23
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/camlp4.rb
==> Dependencies
Required: ocaml ✔, ocamlbuild ✘
==> Options
--HEAD
	Install HEAD version
==> Analytics
install: 359 (30 days), 1,248 (90 days), 20,219 (365 days)
install_on_request: 106 (30 days), 307 (90 days), 1,593 (365 days)
build_error: 0 (30 days)

And:

$ opam switch
#  switch          compiler                       description
→  4.07.1+flambda  ocaml-variants.4.07.1+flambda  4.07.1+flambda
   system          ocaml-system.4.07.0

But bafflingly, nobody seems to be using the system camlp4. I don't remember installing it, but August of last year was when I was doing a bunch of build/docker work, so maybe I did it?

$ brew uses --installed camlp4
$ brew uses --installed ocaml
bibtex2html    camlp4    hevea    ocamlbuild    # output edited for width

If you think the system camlp4 is the problem, is there something we could do to prevent this situation? I'm starting to think that upstream systems (brew, apt, etc.) should list OPAM as a conflict with all OCaml packages other than the system OCaml used to bootstrap OPAM.

[avsm]

This isn't a conflict with opam and system-installed libraries. The camlp4 package in opam is careful to detect if a system one is supplied, and install that one if available. The underlying problem is simply that the upstream packaging is broken -- it's (for decades) been the case that an upgrade of the OCaml compiler requires a recompile of dependent libraries, and Homebrew missed that in one upgrade. @Blaisorblade also just added a test case there to spot this (Homebrew/homebrew-core#39468), so you may have been unlucky with when you upgraded.

There's little that opam can do if the system-supplied versions are inconsistent, beyond installing local overrides (which is what happened when you did the reinstall, I expect).

Hope this helps; since your issue is fixed and we think the underlying Homebrew issue is also sorted, I'll close this issue.

[mgree]

I see! I'm not certain it's quite that simple, though... why is ExtUnix's build failing if the script calls brew update (https://github.com/ocaml/ocaml-ci-scripts/blob/4466db9737c1e615d4aab631bb32b3505ea68ce6/.travis-ocaml.sh#L32)? Recent runs in my own fork don't seem to work, either... do you think this is a matter of waiting for things to percolate, or is something else going on?

[Blaisorblade]

@mgree Your latest CI build was two days ago, that's too old; please try again.

That CI failed because the fix had not yet been committed (if I had written it at all). In particular, https://travis-ci.org/ygrek/extunix/jobs/512680112#L1245 shows version 4.07+1 installed. The fix is in 4.07+1_1.

If you upgrade now, you should get 4.07+1_1, which was rebuilt on latest OCaml. I just confirmed that works for me on macOS High Sierra (the same version as in that CI script), by upgrading and running opam install ulex; other bottles were upgraded too.

[Blaisorblade]

And re your local build, I think the fix was pushed 4-5 hours ago.

[mgree]

Ah: it looks good now. Thank you so much! Glad to know this was transient breakage and not something deeper.