mdx docker codeblock

[avsm]

I'd like to have installation instructions run "live", so that we can regression test our installation instructions in RWO and ocaml.org.

For that, we would need to have some way to redirect a codeblock into docker. E.g. something like:

# apk add --update opam
# opam init -ya
# opam install rwo

And have that execute the commands and fill them in.

[Julow]

The use case is to test their output only ?
Or should it be able to run commands in an other block in the same environment ?
Or even OCaml code ? (This requires a lot of work)

Also, downloading an image and initializing opam will take a very long time and commands that can run automatically may be different than commands run by users. (eg. -y --quiet)
So if this is the main use case, wouldn't it be better to test that somewhere else than in the build of rwo ? (Eg. in a global CI run for every ocaml releases)

[avsm]

All good questions -- I had in mind that this would run less often than other tests. It would be useful to be able to classify some mdx blocks with a tag, and only run those tags (e.g. this could be an install tag in the block).

The usecase is to test their output only. We have installation instructions for >10 distros now, so automating it is extremely useful, especially to see when they start failing. The output could be considered nondeterministic, as with benchmarks.