Add support for building with Nix

[RyanGibb]

This PR adds support for building the project with Nix.

It supports building a binary program as well as a unikernel with https://github.com/tweag/opam-nix and https://github.com/RyanGibb/hillingar respectively.

To build a binary, one can use $ nix build . or $ nix build .#scope.mirageio-bin.

To build a unikernel $ nix build .#unikernel.<target>, e.g. $ nix build .#unikernel.unix.

One can also create a development environment from these derivations. So instead of a new user having to go through the manual process of creating an opam switch and installing all the dependencies (along with debugging steps that might arise with the pinned packages, vendored dependencies, and external system packages) they can just run nix develop .#<derivation> -c <command>, e.g. nix develop . -c codium `pwd` or nix develop .#unikernel.unix -c dune build.

I hope this will enable reproducible builds to help alleviate issues such as:

• struggle with compiling it locally #792
• Build error while running make build #757

[RyanGibb]

I think the CI is failing because dream is using the wrong version of mirage-crypto-rng after mirage-crypto-rng.lwt was split into mirage-crypto-rng-lwt: mirage/mirage-crypto@0b732b7

#=== ERROR while compiling dream.dev ==========================================#
# context     2.1.4 | linux/x86_64 | ocaml-base-compiler.5.0.0 | pinned(git+https://github.com/TheLortex/dream.git#e8ba8e4c6c8689e0e568153fd908369fb8303d53#e8ba8e4c)
# path        ~/.opam/5.0/.opam-switch/build/dream.dev
# command     ~/.opam/5.0/bin/dune build -p dream -j 71
# exit-code   1
# env-file    ~/.opam/log/dream-1-1d9bf4.env
# output-file ~/.opam/log/dream-1-1d9bf4.out
### output ###
# File "src/dune", line 19, characters 2-23:
# 19 |   mirage-crypto-rng.lwt
#        ^^^^^^^^^^^^^^^^^^^^^
# Error: Library "mirage-crypto-rng.lwt" not found.
# -> required by library "dream" in _build/default/src
# -> required by _build/default/META.dream
# -> required by _build/install/default/lib/dream/META
# -> required by _build/default/dream.install
# -> required by alias install

Pinning mirage-crypto-rng = "0.10.7"; fixes this, but I think updating the pinned for of dream is the way to go because TheLortex/dream@fe59e41 fixed this.

[RyanGibb]

This was removed to allow the build to succeed. See #796.

[hannesm]

See #795 for the mirage-crypto-rng 0.11.0 change (and requirements to get this through, i.e. a release of mirage).

I'm sceptical of adding 360 lines to this project (adding to the complexity of it), even more since this code is not covered by CI, and there's no obvious path how to upgrade the lock file (esp. without "nix" tooling - I would be sad if "contributing to mirage-www means you have to have nix tooling and some expertise in that).

This won't fix #792 (for me), since the underlying issue that tailwindcss is distributed as binary for some platforms (not including FreeBSD).

So, thanks for your contribution. I'm happy to enable people to build MirageOS unikernels with NixOS, but I don't think spreading lock files around is a viable path (e.g. we have this in albatross (https://github.com/roburio/albatross) - even with a CI job that attempts to build it - but there's lack of automation on updating the lock file).

I personally like to build (and see failures and fix them) with the latest HEAD of opam-repository, instead of pointing to specific commits and find out weeks, months, years later that there's a huge amount of technical debt accumulated. But that's of course a debatable development methodology.

[RyanGibb]

I'm sceptical of adding 360 lines to this project (adding to the complexity of it) ... I would be sad if "contributing to mirage-www means you have to have nix tooling and some expertise in that).

That is a valid concern. The nice thing about opam-nix is that it continues to work with the existing opam tooling, so it allows those using opam to collaborate with those using nix.

This won't fix #792 (for me), since the underlying issue that tailwindcss is distributed as binary for some platforms (not including FreeBSD).

I believe 434508c should remedy this. It builds tailwindcss from source, and Nix is available for FreeBSD (although I've never used it myself). Nix really shines in handling these kind of multi-language environments.

So, thanks for your contribution. I'm happy to enable people to build MirageOS unikernels with NixOS, but I don't think spreading lock files around is a viable path (e.g. we have this in albatross (https://github.com/roburio/albatross) - even with a CI job that attempts to build it - but there's lack of automation on updating the lock file).

Regarding the lack of support for automatic upgrading of lockfiles, this is perhaps an intentional feature of Nix providing reproducibility. I imagine it would be possible to set up CI to automatically upgrade this, but any breaking changes would need to be dealt with manually (but this is also the case outside of Nix).

I personally like to build (and see failures and fix them) with the latest HEAD of opam-repository, instead of pointing to specific commits and find out weeks, months, years later that there's a huge amount of technical debt accumulated. But that's of course a debatable development methodology.

One can in fact do this with opam-nix too. Removing ocaml-base-compiler = "*" from the query should get you the trunk which you can evaluate with --impure due to the lack of a static hash. I've submitted this PR without the latest head in order to enable reproducibility, however.

[hannesm]

Thanks for your reply @RyanGibb, still I've a couple of questions unanswered:

• why should I buy into nix and try it out on FreeBSD? I'm already at the complexity budget using opam, mirage, opam-monorepo -- now on top I should use nix? if it would remove some of the complexity of the other moving parts, that would be nice, but as far as I understand it just adds another layer
• the file you would like to add is not tested by any CI system (neither the deployment CI, nor the ocaml-ci use nix)
• it is still unclear how this lock file would be updated

I understand that nix uses lock files for reproducible builds and binaries, but as I tried to emphasize in the OCaml & opam ecosystem there are rolling releases of sources, and I find it crucial to stay on top of the releases. I've worked on a thin extension for opam to feature reproducible builds (https://github.com/roburio/orb -- https://builds.robur.coop), which uses the HEAD of opam-repository (and indeed there are lots of updates to opam-repository that don't result in different hashes of the resulting unikernel binaries) -- so there's no requirement to update every now and then commit hashes -- but instead failures can be seen early (and then fixed).

So, I wonder how to join forces or document/explain more clearly what the development model of the mirage-www unikernel (and other unikernels) is -- and from what I read there is a divergence between "let's stick to a certain opam-repository commit" and "let's try and fail the latest, observe CI and fix the code".

On a more technical level, in your lock file, there is twice opam-overlays (and opam-overlays2) with different rev, also twice opam2json (and opam2json2) -- this time with the same rev -- are you aware of that? Is that intentional?

[hannesm]

And to be clear, I'm open to have this merged if others in the @mirage/core team think this is a good path to have. I'm curious what others think in terms of development model for MirageOS unikernels. I'm just hesitant since several years ago it was "let's add a debian subdirectory with changelog, dependencies, ...", then some years ago it was "let's add Dockerfile into each repository", and now it is lock files for various systems (nix, npm, esy, ...).

[dinosaure]

I would like to extend the @hannesm's point of view by the current state of mirage-www. The project is currently not in a good shape when it requires a certain fork of some libraries that we don't have the time to maintain (this is not a criticism here, as I also fit in with those who do not have the time to maintain such forks). This means above all that, as it stands, mirage-www already has serious problems in maintaining itself and in the time available to everyone to keep the project usable.

Adding support for NixOS implies not only a responsibility on your part to maintain that support but also a collective responsibility to those involved in the MirageOS project to keep an eye on that support.

This is, I think, where things become problematic and can be misinterpreted. Like @hannesm, I am not against NixOS support and we have repeatedly shown good signals to the NixOS community to improve MirageOS libraries and software (like Solo5). However, we are not yet in a position to systematically (and for obvious reasons of time) integrate NixOS issues into all our projects.

What would be interesting in this context is to be able to support NixOS semi-automatically from a well-defined tool or workflow - note that MirageOS has already experimented with this kind of transition in the past (notably in the support of AFL). If such a thing exists and can be easily appropriated collectively, I would personally be happy to experiment/test and go further in this direction.

[RyanGibb]

Hi @hannesm and @dinosaure. First of all, thanks for taking the time and the energy to respond.

To respond to @hannesm first:

still I've a couple of questions unanswered:

• why should I buy into nix and try it out on FreeBSD? I'm already at the complexity budget using opam, mirage, opam-monorepo -- now on top I should use nix? if it would remove some of the complexity of the other moving parts, that would be nice, but as far as I understand it just adds another layer

I should preface this by saying that while I find it to be very useful, I'm not trying to convince you yourself to use Nix if you don't have the time or energy to learn this new tool.

I suppose the simplest answer would be that it addresses your issue of building tailwindcss from source. I would also argue that instead of adding another layer on top of all these tools it instead replaces a lot of functionality of Opam with a philosophy I find easier to work with. Rather than creating and maintaining mutable Opam switches, you can instead use Nix to deterministically build and provide sources for development and builds. You can transparent share these dependencies between projects, garbage collect them, and recreate them trivially. I would say it remove a lot of the complexity of Opam; the only part of Opam that is used for in opam-nix is solving the dependency constraints that .opam files provide.

While I find it to be very useful, people like tools they are used to, and it is unarguably additional work to gain familiarity with Nix. I would argue not because it's necessarily more complex but rather because it's unfamiliar.

• the file you would like to add is not tested by any CI system (neither the deployment CI, nor the ocaml-ci use nix)

I don't have any experience with mirage-www's CI, but I could look into setting something up. This would perhaps also help address:

• it is still unclear how this lock file would be updated

I understand that nix uses lock files for reproducible builds and binaries, but as I tried to emphasize in the OCaml & opam ecosystem there are rolling releases of sources, and I find it crucial to stay on top of the releases. I've worked on a thin extension for opam to feature reproducible builds (https://github.com/roburio/orb -- https://builds.robur.coop/), which uses the HEAD of opam-repository (and indeed there are lots of updates to opam-repository that don't result in different hashes of the resulting unikernel binaries) -- so there's no requirement to update every now and then commit hashes -- but instead failures can be seen early (and then fixed).

It seems like a CI action or GitHub action that runs nix flake update (or nix flake lock --update-input opam-repository) could provide the benefits of both. The benefit of having such a lockfile in your project is that you essentially track the external dependencies to the exact hashes along with the source code. You could in fact not commit the lockfile and recreate it on every build, but this might lead to incompatibilities if breaking changes were made to one of the flake inputs (e.g. opam-nix).

So, I wonder how to join forces or document/explain more clearly what the development model of the mirage-www unikernel (and other unikernels) is -- and from what I read there is a divergence between "let's stick to a certain opam-repository commit" and "let's try and fail the latest, observe CI and fix the code".

I am afraid I'm unfamiliar with how other unikernels are developed in general. I can certainly see the advantage to the former though.

On a more technical level, in your lock file, there is twice opam-overlays (and opam-overlays2) with different rev, also twice opam2json (and opam2json2) -- this time with the same rev -- are you aware of that? Is that intentional?

This is the result of different projects lockfiles composing. By default it's conservative and duplicates them, but it's possible to deduplicate them. I'll push an update doing this now.

---

And to respond to @dinosaure:

I would like to extend the @hannesm's point of view by the current state of mirage-www. The project is currently not in a good shape when it requires a certain fork of some libraries that we don't have the time to maintain (this is not a criticism here, as I also fit in with those who do not have the time to maintain such forks). This means above all that, as it stands, mirage-www already has serious problems in maintaining itself and in the time available to everyone to keep the project usable.

I understand your problem, and empathise with it.

Adding support for NixOS implies not only a responsibility on your part to maintain that support but also a collective responsibility to those involved in the MirageOS project to keep an eye on that support.

This is, I think, where things become problematic and can be misinterpreted. Like @hannesm, I am not against NixOS support and we have repeatedly shown good signals to the NixOS community to improve MirageOS libraries and software (like Solo5). However, we are not yet in a position to systematically (and for obvious reasons of time) integrate NixOS issues into all our projects.

My motivation in this PR is not to add support for NixOS (or else I would have also added a NixOS module for deployment on a NixOS system), but rather simply support to build the project with Nix for the benefits in reproducible builds and development environments. Nix, for our purposes here, is a build system and package manager. And NixOS is a Linux distribution built on top of Nix. I've written a bit about the differences between Nix and NixOS. However, I would be happy to help to maintain this Nix support.

I hope this clarifies that I do not aim to support an additional operating system with this, but rather integrate a useful tool.

What would be interesting in this context is to be able to support NixOS semi-automatically from a well-defined tool or workflow - note that MirageOS has already experimented with this kind of transition in the past (notably in the support of AFL). If such a thing exists and can be easily appropriated collectively, I would personally be happy to experiment/test and go further in this direction.

Regarding semi-automatic support, I'm not entirely sure how this would work. I think the closest existing project is https://github.com/RyanGibb/hillingar which aims to be an interface to simplify adding support for building Mirage unikernels with Nix. This flake.nix is really just a simply wrapper calling functionality in hillingar for building a unikernel and opam-nix for building a package and providing a development environment, alongside some fixes to work around quirks of the project.

In summary, I think the benefits of this PR is support for reproducible development environments and deployments.

[RyanGibb]

Reading leanprover/lake#15 (comment) made me realise that an alternative might be just to omit committing the lockfile.

[RyanGibb]

However I can see there isn't much interest in this so I'm going to close the issue