Testing: reduce amount of logs produced, drop ounit2

[psafont]

Using alcotest allows us to ignore the output of the scale tests which produced more than 8 thousand lines, for no good reason.

Dune now generated http-lib and rrd-transport, which allows dropping the dependecy on ounit in 3 packages

As always, best reviewed commit-by-commit

[psafont]

There's a test failing in CI, but it's not being reported (!)

  Testing `Metrics scalability'.
  This run has ID `WMVA73ZI'.
  
  ..F
  
  ┌──────────────────────────────────────────────────────────────────────────────┐
  │ [FAIL]        V2          0   Write and read.                                │
  └──────────────────────────────────────────────────────────────────────────────┘
  [exception] Sys_error("/dev/shm/test-metrics3b5500.tmp: File exists")
              Re-raised at Stdlib__Filename.temp_file.try_name in file "filename.ml", line 348, characters 30-37
              Called from Dune__exe__Test_common.make_shared_file in file "ocaml/xcp-rrdd/test/transport/test_common.ml" (inlined), line 145, characters 2-63
              Called from Dune__exe__Test_scale.run_tests.(fun) in file "ocaml/xcp-rrdd/test/transport/test_scale.ml", line 86, characters 23-42
              Called from Stdlib__List.init_aux in file "list.ml", line 69, characters 12-15

This seems to be a (hopefully) very rare name clash. I've threaded a number so it's impossible to make the names clash when generating a large amount of temp files

[edwintorok]

If this is causing test failures it might be because there are leftovers from a previous test, or we run multiple tests in parallel and they clash on the names. Filename.temp_file tries 1000 times to generate a name (using only 6 random hex digits though).

We could try to make this more unique by including the PID perhaps? And maybe register an at_exit handler to safe unlink them, so even if the test fails/gets interrupted we'd clean it up.

[edwintorok]

We could also increment a (global) number ourselves to make this more unique. I see we generate 4096 files in a test, so we could include PID + globally incrementing number in the suffix.

[psafont]

There could be a conflict between test_scale and test_unit when run simultaneously. Adding the number like I did would remove the possibility of conflict. We could add the name of the executable, or pid (as you say), or even function. But I don't see the point with the current solution

[edwintorok]

Pinning packages in the CI is very very slow and requires a lot of disk space, and should be rarely needed, as you say only when new depexts get introduced, did you find a case where this caused a failure?
Would be good if opam supported determining depexts too without pinning.

In general we should probably reduce the number of packages we have if we want to pin (we can keep them as separate dune libraries still), but that would be the topic for a separate PR.

[psafont]

It's needed when adding a new package, because it's not present in the existin xs-opam.

When opam needs to install the depexts, it tries to resolve the dependencies of all the present packages. This fails because the package does not exist in xs-opam.

I agree that this is rare, and pinning makes the CI builds very slow. Using opam 2.1 in CI should make it somewhat faster, but it's still not great and it's nopt yet ready. Alternatively I can remove the pinning once the xapi-sdk package is added to xs-opam, and reenable pinning when / if we need to add a new package.

Another alternative can be keeping it disabled here, but add the package (without dependencies) in xs-opam, this allows the depext to work in xen-api. After the xen-api change is merged, xs-opam might start failing, but it can be fixed by updating the metadata from xen-api.

I think the latter is usually better, but currently all xs-opam builds are broken until the sdk package gets added, so we're in a egg-and-chicken situation, and the pinning unblocks us