Blocking issue with the conf-libssl.3 dependency in a macOS/brew environment

[erikmd]

Hi; since yesterday, we experience a blocking issue with a macOS build on GHA (direct link to the failing log).

If I'm not mistaken (I'm not a macOS power user), this comes from the fact:

• there are two flavors of openssl in brew,
  • https://formulae.brew.sh/formula/openssl@1.1
  • https://formulae.brew.sh/formula/openssl@3
• and https://formulae.brew.sh/formula/openssl used to be an alias for https://formulae.brew.sh/formula/openssl@1.1
• but this alias has been removed very recently, so that https://formulae.brew.sh/formula/openssl now results in a 404 error.

So I believe https://github.com/ocaml/opam-repository/blob/master/packages/conf-libssl/conf-libssl.3/files/homebrew.sh should be updated (directly in conf-libssl.3 or maybe in conf-libssl.4?)
I'm not 100% sure but maybe the script should accept the two different openssl@* packages…

Cc @kit-ty-kate (and @yurug @AltGr FYI)

[erikmd]

BTW I guess that the depext spec

opam-repository/packages/conf-libssl/conf-libssl.3/opam

Line 28 in f8dfc4e

["openssl"] {os-distribution = "homebrew"}

should also be replaced with ("openssl@1.1" or) "openssl@3".

[idkjs]

Try

brew install openssl@3

seems to work for me.

[erikmd]

Try

brew install openssl@3

seems to work for me.

Yes, this is the standard workaround. But ideally, this should be done automatically if one sets OPAMDEPEXTYES=1 (which is not the case currently because of this bug)

[kit-ty-kate]

I've tried locally and brew install openssl just installs openssl@3 for me and $(brew --prefix openssl)/lib/pkgconfig contains openssl.pc. opam install conf-libssl also works fine so I'm not sure what went wrong in your GHA run.

[smorimoto]

As an ex-Homebrew maintainer, I don't think they're going to delete the openssl alias, and we can actually see that it's always been here: https://github.com/Homebrew/homebrew-core/blob/master/Aliases/openssl
I don't know why the error happened, even so, I still have some possibilities in mind. But it's all pretty rare edge cases.

[erikmd]

Hi all, thanks for your answers!

I'm not sure what went wrong in your GHA run.

Okay! BTW I had workarounded the "issue" after opening #19676 by manually adding a brew install openssl@3 command;
so maybe this one is actually unnecessary because the issue was just a heisenbug 😅

So I'll close this issue for now (but I'll reopen / comment here if ever we spotted another conf-libssl installation failure…)

[kit-ty-kate]

Okay! BTW I had workarounded the "issue" after opening #19676 by manually adding a brew install openssl@3 command;

I'm guessing the reason why it worked is that the brew install command does brew update from time to time, whereas opam does not to keep the experience consistent (https://github.com/ocaml/opam/blob/adbe57b006467ed8f173bdbcdb14b8891cef5e98/src/state/opamSysInteract.ml#L650), and the homebrew repository state must have been in an in-between state where the openssl package was half broken.

I would suggest adding opam update --depexts instead to prevent this kind of issue in the future. (opam depext -u if you still use opam 2.0)

[erikmd]

Hi @kit-ty-kate, FYI I applied your suggestion here:

ocaml-sf/learn-ocaml#454

but I got a failure :-/

https://github.com/ocaml-sf/learn-ocaml/runs/4086278147?check_suite_focus=true

[kit-ty-kate]

Mmmh, I got the same issue locally but I’m a bit surprised by what Homebrew is telling me:

$ brew --prefix openssl
/opt/homebrew/opt/openssl@3
$ ls /opt/homebrew/opt/openssl@3
ls: /opt/homebrew/opt/openssl@3: No such file or directory

This is fixed by brew install openssl.

Reading the homebrew manpage and testing a few things it seems that brew --prefix does not do what we want. brew --prefix <formula> does not check at all if the package is installed and only return what the path will be if the package is installed.

brew --prefix --installed <formula> brings us one inch closer by erroring out if the package is not installed. However there is still an issue: if openssl@1.1 is installed brew --prefix --installed openssl will fail because it translates openssl to be an alias for openssl@3 and not the other version.. and opam thinks openssl is installed because openssl@1.1 is installed.

The problem could be resolved if we had a way to know which package is considered an alias of another one but going through the Homebrew manpage there doesn’t seem to be a simple way.
If we had that we could fix opam to install the correct package.

Alternatively we could reimplement brew --prefix <formula> ourselves in opam-repository by using brew --prefix without argument and scanning the directory for the latest installed package

[erikmd]

Hi @kit-ty-kate,
OK thanks for your feedback and tests!

But I was thinking that having openssl@1.1 should still be OK regarding the conf-libssl installation test (?)

I mean, in this case we could have:

• opam install conf-libssl.3's install test return OK if openssl@1.1 or openssl@3 is installed,
• opam install conf-libssl.3's depext installs openssl@3 if none of the version is installed.

WDYT?

(again, I'm not Homebrew-savvy so this is just a quick guess 😅)

[erikmd]

Hello, I think this issue should not be closed yet / let me know if you need more tests or so

[samoht]

@kit-ty-kate do you think it would be possible to add openssl@3 as a depext for homebrew? Otherwise it's still looks pretty broken?