Add doc about making brew package with dune pkg

[gridbugs]

No description provided.

[Leonidas-from-XIV]

This guide is an interesting idea on how to leverage dune pkg for system packages. I haven't thought of this before but it is quite neat that you only need to depend on dune.

[Leonidas-from-XIV]

But there are practically no packages that do this?

[gridbugs]

Not yet, but hopefully once package management is released packages will start doing this.

[Leonidas-from-XIV]

It seems weird to me to have a guide that's not applicable to anyone? Especially given that we're mostly pushing for autolocking, so even with stable dune package management I would expect most packages to not use a lock.

[gridbugs]

What if we framed locking the project as part of the guide, rather than as a prerequisite for following the guide. I suppose the problem with that would be that all the existing released source archives still won't have a lockdir in them, but I think it's ok that this be a guide for people who want the next release of their project to be available on homebrew. I think it's risky to encourage people to make homebrew packages that solve dependencies as part of the installation process.

[Leonidas-from-XIV]

The issue with locking as part of packaging is that the packager would need to effectively fork upstream and use a custom tarball which will most like raise eyebrows for homebrew maintainers, especially given the situation of compromised supply chains in FOSS…

I guess a rather large patch that adds the lock dir contents to the package could work however and be practical enough to work on most OCaml software in the wild.

[gridbugs]

Ah yes I hadn't considered the use case where someone other than the project maintainer is adding the homebrew package, and the project doesn't have a checked-in lockdir. I intended this to be a guide for publishing one's own package on homebrew, not someone else's package.

As you correctly point out, it would require the author of the homebrew package to lock the project themselves, and have the homebrew package download a custom tarball which contains the lockdir, which does look suspicious.

I'll add an explanation for how to have homebrew solve dependencies at install time.

[Leonidas-from-XIV]

Adding

system "dune", "pkg", "lock"

here could make this work for nearly any OCaml package that uses Dune. To me this seems more practical as it does not require the tarball to ship with lock files that support the platform that the user is trying to package for.

[gridbugs]

I think the problem with that approach is that builds won't be reproducible if the source archive doesn't include a lockdir. My preference would be for the guide to instruct how to release a homebrew package that never fails to build due to solver errors, and always builds the same artifacts on any given platform.

[shonfeder]

Since reproducible builds are a goal for homebrew encouraging an approach which support reproducibility seems advisable?

However, since presumably homebrew dependencies can be updated between builds, I'm not sure how much this is goal is actually achieved in practice in homebrew.

If we had a flag like dune build --pkg-enabled or could set the envar with DUNE_CONFIG__PKG_ENABLED=true dune build, then we could have a single instruction set, that would work with a checked in lock dir if present, or fallback to a lockless dependency management otherwise, right?

[shonfeder]

Reads pretty good to me! Awesome to have this documented!

[gridbugs]

Based on @Leonidas-from-XIV's feedback I've updated the doc to suggest locking the dependencies at install time.

[shonfeder]

Won't this force projects to update their locks, so even if you have a checked in lock dir it will force deps to be updated? That would be very bad behavior, IMO.

[shonfeder]

If we cannot fix this in a clean way like #12744 (comment) (that would work correctly with either a lock checked or without a lock, without a conditional command), then I suggest:

1. We plan to add a feature that will allow a clean command (adding an issue accordingly).
2. For this PR, just note in the set up that having a checked in lock directory OR a workspace with (pkg enabled) is currently a prerequisite for this tutorial.

[gridbugs]

The doc says

Note the first install command: dune pkg lock. This is only necessary if the source archive doesn't contain a lockdir...

But yes, also happy to say that for now projects either need a checked-in lockdir or (pkg enabled) in dune-workspace and remove dune pkg lock from the install commands.

[shonfeder]

Noting a change request, because I think forcing an unconditional relock in build recipes could have an untenable behavior.

[gridbugs]

After speaking with @shonfeder, I've changed the example formula to include system "dune", "pkg", "lock" commented out, with an explanation that it should only be uncommented if the source archive lacks a lockdir.