Add a stage to lint documentation errors

[craigfe]

This resolves #94.

This stage follows the same structure as the existing lint step, but shares some logic with the Opam build steps in order to install the appropriate dependencies before building the documentation.

[craigfe]

Rebased over #108 to get the CI working.

[craigfe]

Rebased again. @talex5 / @avsm, can I get another review?

[talex5]

CI is failing...

[craigfe]

Thanks; bad rebase. Should be fixed now.

[talex5]

This failed on the first two packages I tried it on, e.g.

#10 [6/8] RUN opam depext --update opam-zi.dev
#10 5.608 # Detecting depexts using vars: arch=x86_64, os=linux, os-distribution=alpine, os-family=alpine
#10 8.503 [ERROR] No solution for opam-zi.dev: Your request can't be satisfied:
#10 8.503           - unknown package

How are you testing it? I'm using ocaml-ci-local.

I think we should be using the same code to generate the Dockerfile installation lines for the lint stages as for the main build stages.

Is odoc just a binary? Perhaps we could build that in a separate Docker build stage and then COPY --from=odoc .... Then we could cache both the building of the project and the building of odoc.

[craigfe]

I've rebased and changed the approach slightly. The logic for installing Opam dependencies of the project is now properly shared with opam_build.ml by exposing some more logic inside that module.

Tested to be working on Alcotest, Irmin and OCurrent in ocaml-ci-local mode.

[talex5]

I've deployed it... let's see what happens!

[avsm]

Odoc is just a binary, and is installed that way in the duniverse_build.ml (install_bin). @craigfe, why is the doc target disabled in the duniverse build? The intention is very much to build docs there, so that the stage can be further extended in the future to a deployment/staging branch for visual inspection.

[craigfe]

The intention is very much to build docs there, so that the stage can be further extended in the future to a deployment/staging branch for visual inspection.

@avsm: I'm keen for that feature too, but it's not clear to me why it should apply only to Duniverse projects. Shouldn't Opam builds get a deployment/staging branch for their docs too? That's why I proposed taking it out of duniverse_build.ml: right now it seems like duplicated functionality unique to Duniverse projects.

[avsm]

But you've taken out the feature without adding it anywhere else for them. Remember that right now, duniverse builds are distinct from opam builds. I'm happy to refactor it when we add the staging support, but right now I'd like to ensure that it stays building correctly with duniverse projects.

[craigfe]

This linting step should still run for Duniverse builds currently, no? These jobs are defined outside of the regular build matrix.

It seems to be running correctly on e.g. realworldocaml/book:

[talex5]

For me, book is now failing:

#125 4.218 The following dependencies couldn't be met:
#125 4.218   - rwo -> ocaml >= 4.10.0
#125 4.218       base of this switch (use `--unlock-base' to force)

https://ci.ocamllabs.io:8100/job/2020-04-21/184252-docker-build-be7e04

[avsm]

Right, that'll fail because the duniverse build is different from the opam build. The doc linting step is trying to copy all the opam files, and to pin them, and then run depexts, and then install them, etc. The duniverse build doesn't install opam deps, and uses the locally vendored copies instead with a single dune build.

There's a material difference in how the opam and the duniverse builds work. I think in the interests of pushing this PR forward, it should leave them distinct, and we can work on converging them in a next step.

[craigfe]

Linting steps are now disabled for Duniverse projects, and I've reverted the change to not build docs during the build stages of Duniverse project.

The current solution uses a bind to conditionally execute linting according to the results of the analysis step.