opam-builder

Monitor and build the opam-repository, and display results on the web. It uses a patched version of OPAM to avoid the recompilation of already compiled packages (binary packages), and is completely incremental, i.e. not recomputing anything that has not changed since the last commit.

Currently, the results of running opam-builder on a 4-core server running Debian 8 are displayed here:

http://opam.ocamlpro.com/builder/html/report-last.html

Build

Dependencies

The following dependencies are needed:

• ocp-build >= 1.99.10-beta
• OCaml >= 4.00
• aspcud (at least the version of Debian 8)

Compilation

Then, run the following:

./configure
make

Patched version of OPAM

You also need a patched version of opam. Use the branch:

https://github.com/lefessan/opam/tree/2016-03-02-opam-builder

You should compile this patched version of opam, and install the resulting binary in your PATH under the name opam.dev.

Depexts

Since you are going to build the entire opam-repository, you should make sure you have all the external dependencies available. This is usually done by calling opam depexts. You might want to do it using your standard OPAM setting before installing opam-builder.

Usage

opam-builder has 3 different modes:

• build: the default mode is to monitor a git clone of the opam-repository and try to build the commits for a given OCaml version
• lint: in lint mode, opam-builder monitors a git clone of the opam-repository and calls opam lint on every modified opam file after a commit
• import: scan the reports generated by the build and lint modes, and generate webpages to display the results

In all 3 modes, opam-builder does not exit, it will run forever, watching either git commits or report directories.

Build mode

Suppose we want to run opam-builder for version 4.02.3 in a directory $HOME/opam-builder:

First, clone the opam-repository:

cd $HOME/opam-builder
git clone git@github.com:ocaml/opam-repository.git 4.02.3

opam-builder assumes the remote for the official repository is called ocaml:

cd $HOME/opam-builder/4.02.3
git remote add ocaml git@github.com:ocaml/opam-repository.git

Now, we will call opam-builder in the cloned repository, telling it that we want to create an opam switch for 4.02.3 within that repository (i.e. the root of OPAM will be 4.02.3/.opam).

cd $HOME/opam-builder/4.02.3
nohup $HOME/opam-builder/sources/_obuild/opam-builder/opam-builder.asm 4.02.3 &
tail -f nohup.out

Not that opam-builder will look a patched version of opam called opam.dev, as explained before.

While opam-builder is installing the switch (and before it finishes), you MUST edit the opam config:

gedit $HOME/opam-builder/4.02.3/.opam/config

and set the number of jobs to 1. IT's VERY IMPORTANT.

If you don't do that, the results of opam-builder are meaningless, as opam is going to parallelize the compilation, without regards to the fact that opam-builder is snapshotting the switch.

Normally, from now on, opam-builder is going to run and monitor the git repository. That's the reason why we used nohup to start it, as you probably want to log out from your server, keeping opam-builder running in the background.

After compiling the whole repository (the first time), opam-builder will only rebuild the packages that have been changed by the last commit, so even if the first time takes a few hours (or days), new commits should compile much faster.

opam-builder generates reports in a sub-directory called reports.

Lint mode

Follow the same procedure as the build mode, but call opam-builder with the command:

cd $HOME/opam-builder/repo-lint
nohup $HOME/opam-builder/sources/_obuild/opam-builder/opam-builder.asm --lint-only &
tail -f nohup.out

Import mode

Once you have setup several build nodes and a lint node, you can use the import mode to generate the result pages (you should probably wait for at least one node to complete a full build before generating the first page):

cd $HOME/opam-builder
nohup  $HOME/opam-builder/sources/_obuild/opam-builder/opam-builder.asm \
  --import \
  $HOME/opam-builder/repo-lint/reports \
  $HOME/opam-builder/4.02.3/reports \
  $HOME/opam-builder/4.01.0/reports \
  $HOME/opam-builder/4.00.1/reports &
tail -f nohup.out

It should create a sub-directory html/ containing a hierarchy of directories and the report-last.html file corresponding to the last commits.