Adopt scala-native-config-brew

[armanbilge]

https://github.com/armanbilge/scala-native-config-brew

It doesn't force anyone to use brew, but it will auto-configure everything if they have installed s2n-tls with brew.

Edit: also, this is not transitive. Only applies to our build.

[rossabaker]

Does anything pin the brew dependencies? I am concerned about reproducibility, before and after this PR.

[armanbilge]

No, nothing pins the dependencies 😓 I don't think it's really possible to pin dependencies independently with brew.

I guess this is something Nix might be better at? Maybe time to learn that 😅

[armanbilge]

So I did some research on Nix. Ok, I'm sold. They had me at the hashing scheme for /nix/store.

Based on my newly-acquired (mis-)understanding:

Now, it would be one thing if we were running our entire CI inside of a Nix shell. Is this easily possible? Then we'd be reproducible all the way down.

If we can't do that ... is there a good way to use nix as a package manager without using the nix shell? Seems like it could be possible, but not nearly so nice as the shell.

The sad truth is that at the end of the day our CI is fundamentally not reproducible. GitHub is regularly updating their runner images, our JDK index is regularly swapping out JVM versions, etc. Although I guess within the JVM the story is much better.

What we can do is install s2n-tls from source in CI. Then we can pin to a specific version. Although it would still be using whatever OpenSSL, libc, LLVM toolchain, etc. that happen to be available.

[rossabaker]

Yes, a disciplined SBT build (e.g., no snapshots, no weird resolvers) and Nix offer many overlapping benefits, which is why I haven't pushed Nix as the Build Of Truth on Typelevel projects. It's just a development convenience. You correctly note several other impurities in the existing action. We handwave around JVM patches in particular, but they've broken Blaze TLS tests in the past.

Full reproducibility can be achieved with a derivation, materialized with nix build. At work, we use sbt-derivation on applications for a pure Git => Executable. For libraries, one could imagine a pure Git => TargetDir. "Publishing" could be pushing to a distributed cache like Cachix, or shrugging and making everyone materialize it. But to be useful outside Nix, we need a TargetDir => IO[Unit] to publish to Sonatype, and effects are outside the scope of deriviations.

We might instead focus a reproducible toolchain, so we pin SBT and the JVM and the native libraries and git and bash and what not in the flake, invoke SBT, and then pray that whatever we do with it atop that foundation is idempotent. Invoking SBT from CI could then take a few forms:

• nix develop -c 'sbt ...' (to use the SBT in the devShell using the devShell's environment hooks)
• nix run .#sbt -- ... (if we wrap SBT with a script that sets its environment variables)
• nix profile install .#sbt; sbt ... (imperative, disrecommended for long-lived workstations)

I suspect the third option would be the least disruptive in an sbt-typelevel-github-actions world.

[armanbilge]

• Use Nix for CI #6727

[armanbilge]

After discussing with @TimWSpence I am reviving this PR.

The Nix devshell will continue to be used for CI and will be the recommended environment for development.

However, the brew plugin can provide a practical fallback for the not-yet-enlightened :)

[armanbilge]

This will effectively disable the brew plugin when running in the devshell.

[danicheg]

He-he (it doesn't mean something bad, just love this self-promoting 👉👈)

[TimWSpence]

Yeah Arman, stop doing all the work yourself 😛