Skip to content

Mirage 4.0, dune and cross-compilation #1195

Closed
@TheLortex

Description

@TheLortex

Hi, this issue introduces the final changes we need to have dune to build unikernels in mirage.
It comes in the continuation of #969. These changes have been partially implemented for testing purposes, this issue exists to make sure everyone is aware of the update plan.
Feel free to comment on this issue if you need precisions on a specific point !

The changes

The unikernel build is now done in two steps:

  • install a cross-compiler for the desired target via opam.
  • building the unikernel using duniverse and dune.

Step 1: cross-compilers

As most mirage targets actually build for the same architecture as the host, we don't actually need cross-compilation to build unikernels. However it has the advantage of having a simpler mental model. Code that needs to run on the host system are made with the host compiler, and code that needs to be run in the unikernel are built with the target compiler (the cross-compiler). Having first-class cross-compilation support easily enables new targets such as esp32 or risc-v. Ocaml cross-compilation is not a nice story, it's still hard in 4.11.1 to build a cross-compiler: a lot of tweaks have to be done to be able to build one correctly. Therefore the biggest changes are in ocaml-freestanding: it is split in two, libc and cross-compiler, so that the configuration step of the cross-compiler can be done with the installation path of the libc.

This is the summary of changes:

  • Mirage tool is updated, unikernels are either built for unix or cross-compiled. Cross-compiled backends may require to install toolchains with opam (ocaml-freestanding, solo5-bindings-<x>).
  • Introduction of a new, header/tools only solo5-headers opam package.
  • Introduction of solo5-libc package containing nolibc and openlibm.
  • ocaml-freestanding becomes a cross-compiler based on solo5 headers and libc. It is installed in <opam-switch-root>/freestanding-sysroot/, and can be refered to using ocamlfind -toolchain freestanding <command>.

Step 2: duniverse

The opam tooling is not ready for cross-compilation, so we need to rely exclusively on dune to perform unikernel builds. This means that a tool was needed to fetch and download all the required dependencies of the unikernel. This is the role of duniverse.
duniverse, now known as the opam plugin opam monorepo, uses opam to parse the dependency file and locate all the required sources, before downloading them in a ./duniverse folder.
When the sources are fetched, dune is configured to have one build context per mirage target, set up to use the correct compiler for each target. Then a single dune build command is able to build all the required dependencies to produce the requested unikernels.

This is the summary of changes:

  • mirage configure -t <target> generates:

    • dune: build rules
    • dune-project: general dune configuration
    • dune-workspace: definition of build contexts, that's how installed cross-compilers are detected by dune
    • <unikernel>-hvt.opam: opam file definition, declaring dependencies (runtime, build and toolchain). Toolchain dependencies are guarded by a new build-context variable and aim to be installed by opam.
    • mirage.context: command line arguments for mirage configure
    • Makefile
  • Duniverse is now an opam plugin, called opam monorepo. It parses an opam file to compute the transitive closure of a project's dependencies, and locally fetches them. It uses the opam resolver so it supports opam pins and repos.

  • opam monorepo lock: solve the dependencies and find where to fetch each project, generating a <unikernel>.opam.lock file.

  • opam monorepo pull: download the dependencies according to the lockfile.

Workflows

Build a unikernel from scratch

  • Install OCaml and opam
  • opam install mirage
  • mirage configure -t <target>: generates build / install files (see earlier)
  • make depend or mirage depend: install dependencies:
    • toolchains using opam (env OPAMVAR_build_context=1 opam install . --deps-only)
    • using duniverse (equivalent to opam monorepo lock && opam monorepo pull)
  • dune build: build the unikernel for all requested targets, located in _build/mirage-<target>/.

Then, it's possible to:

  • change unikernel.ml to modify the app: dune build
  • change config.ml to change the configuration:
    • If there's some dependency change: mirage configure -t <target> && make depends
    • dune build
  • install the unikernel binary: dune install.

Updating libraries

  • A new package is merged in opam - run opam update to get the latest packages in the current switch.
  • To update the duniverse/ dir: mirage configure -t <target> && make depends.
  • dune build to rebuild the unikernel.

Locally editing a package in the dependency tree

  • cd duniverse && rm -rf <package> && opam source <package> (or do a local clone of the corresponding git repository)
  • hack on it (dune build && dune test) at the repository root
  • push your changes upstream and modify config.ml to add a new pin-depends to share the changes with others.

Use a custom opam repository

  • all installed packages must use dune to be built in an unikernel
  • add the repo to the current opam switch opam repo add <name> <url>
  • make depends: duniverse will pick up the newly configured opam repository.

Specific points

Testing packages / CI ?

A problem is the fact that packages destined to be built with duniverse only (such as mirage-solo5) will still be published on opam, and their installation has no meaning anymore. However publishing them on opam means that they still can be tested by the opam CI. Packages that aim to be cross-compiled (thus installed by duniverse) must not have any non-dune dependency. So to test these packages, non-dune dependencies (such as ocaml-freestanding) need to be set as {with-test} dependencies.
I'm currently experimenting with an ocurrent pipeline that uses mirage-dev and mirage-skeleton to test unikernel builds: https://github.com/TheLortex/mirage-ci.

Dune

A big change in the mirage workflow is that the whole unikernel dependency tree must be built with dune. As a consequence, non-dune dependencies have been forked, these forks being picked up by duniverse using an overlay repository: https://github.com/dune-universe/opam-overlays.
For now, we have to maintain forks, but other solutions are to be discussed:

  • perform the switch to dune in the upstream repository.
  • sandbox the build process using dune features.

For a general-purpose library to be mirage-compatible, there are several requirements:

  • It must not depend (runtime) on the unix library or target-specific mirage libraries such as mirage-solo5, mirage-xen.
  • It must not depend (either runtime or {build}) on packages that are not built by dune.
  • C stubs needs to be standalone and not use glibc to ease cross-compilation with mirage targets (e.g. be platform-independent).
  • test dependencies are not constrained because they are installed by opam.

For target-specific mirage libraries, the requirements are the following:

  • It must not depend (either runtime or {build}) on packages that are not built by dune.
  • It may depend on same-target dune-built packages.
  • It may have arbitrary test dependencies.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions