Dedicated support for duniverse builds #142
Conversation
|
I've pushed some changes to my branch at https://github.com/talex5/ocaml-ci/tree/duniverse:
I squashed the changes because it's more useful to see the diff with master than with the previous commit (e.g. there are fewer changes to A test realworldocaml build failed, but it doesn't look like a CI problem, e.g. |
|
That was a real bug; merged realworldocaml/book#3300 to fix it (the dune-get file was out of sync in the main rwo repo). i've force pushed your branch to this PR. |
Why undo this? For opam projects, only the toplevel .opam files should matter. It was only for duniverse projects that we previously needed to scan lower ones (to get depexts). For opam projects, using the toplevel opam files only matches the behaviour of |
|
opam projects with vendored dependencies (including ocaml-ci itself) need to scan for opam files in submodules. |
| ~repo:"ocaml/odoc" ~tag:"1.5.0" | ||
| ~bins:["bin/odoc", "/usr/bin/odoc"] "odoc" in | ||
| let ocamlformat = install_bin ~compiler | ||
| ~repo:"ocaml-ppx/ocamlformat" ~tag:"0.14.0" |
There was a problem hiding this comment.
Hm, this is probably buggy. We need to install the precise ocamlformat version required by the project, and that is found in the analysis phase.
|
Ah I see. Something of a hybrid opam/duniverse workflow there. |
- Add a duniverse builder which does incremental builds without requiring an opam environment. It does this by building all the required platform binaries (dune, odoc, ocamlformat and duniverse) in a multistage build and copying them into the main package build. - Extract only the OCaml versions necessary from the lockfile, and use that for duniverse builds. No solver required to infer this as it is explicit. - Remove the duniverse support from opam builder, so that can be simplified back to its original state. Some changes from talex5: - Rebased onto master and removed binds. - Errors parsing dune-get are now fatal. - Get dune-get from the project, not the cwd. - Skip reading `dune-get` if not a dune project. - Removed maxdepth change. Duniverse doesn't seem to care about opam files anyway, except for detecting ocamlformat in the unit-tests, which were previously broken. - Just split the Dockerfile generation from `opam_build`, not the pipeline. - Removed Dockerfile cache. Shouldn't be needed now with incremental evaluation.
|
I've pushed an ocamlformat sync, so this all passes CI now. I can refine the ocamlformat/duniverse semantics in a future PR. For now merging/deploying this would be useful to let the live CI's on realworldocaml and duniverse start passing again :-) |
|
I've deployed it for testing. There was a slight problem where duniverse projects with no compilers would report failure to GitHub, but not show any failure when you went to the page (because the analysis had passed and there were no failed jobs). I've made the analysis phase fail in that case. |
|
Thanks, that sounds right. I'll work on refining the handling of ocamlformat/odoc/dune versioning constraints in dune-get/duniverse next. |
Add a duniverse builder which does incremental builds without
requiring an opam environment. It does this by building all
the required platform binaries (dune, odoc, ocamlformat and
duniverse) in a multistage build and copying them into the
main package build.
Extract only the OCaml versions necessary from the lockfile,
and use that for duniverse builds. No solver required to infer
this as it is explicit.
Detect duniverses by the presence of a dune-get file, and not
the duniverse/ directory. If the sources aren't checked in, then
duniverse will pull the locked sources as part of the build.
This is all incrementally cached correctly.
Remove the duniverse support from opam builder, so that can
be simplified back to its original state.
There are a few TODOs in here. @talex5 indicated that he could try to get rid of the bind in the analysis phase, so feel free to push to my branch with that...
Two good test repositories with the correct dune-get files are duniverse and realworldocaml/book.