Move docker build from build script to xtask

[divergentdave]

This closes #2081, removing the build script that was calling docker build. A new xtask package is added, and its executable is aliased to run as a cargo subcommand. Now, cargo xtask test-docker will invoke docker buildx bake to build the same images, and pass the image IDs through environment variables to cargo test, rather than including image contents in the source. All tests that depend on the container images are put behind the testcontainer feature flag. Testing in CI invokes cargo test with the testcontainer feature via cargo xtask test-docker-with-images instead, using images loaded from a preceding CI job.

[tgeoghegan]

I'm concerned that we don't run cargo xtask test-docker anywhere in CI (unless I missed it). Maybe if test-docker were broken out into two subcommands, one for building the container images and one that uses them in a test, then the former could be used in the janus_interop_docker job and the latter could be used in janus_build?

Or if we ran cargo xtask test-docker in janus_build after loading the Docker images tarball emitted by janus_interop_docker, would docker buildx bake swiftly notice that the images it wants to build are already present in the local repository and move on, or would it insist on rebuilding them?

[tgeoghegan]

This will upload the three docker images, which are 40ish MB apiece, on every PR build. I'm concerned about our utilization of artifacts storage. The GitHub docs discuss a 90 day default retention period for artifacts, but nothing about a maximum on storage. Maybe we should go ahead with this and see if we start getting angry emails about using up our storage quota?

Or maybe we should preemptively tighten the retention period down to maybe 1 day, since we expect the artifacts emitted by this job to be immediately consumed by the test job.

[tgeoghegan]

Seems easier to put #[cfg(feature = "testcontainer")] at the module level, since I don't think we ever intend to support testing Daphne outside of a container.

[divergentdave]

There is one test, janus_in_process_daphne, that can be run using an in-process Janus leader and a Daphne helper in a container. It's disabled for the time being until we can upgrade Daphne, but I'd like to keep it available with default features.

[tgeoghegan]

🎉 🙌🏻

[branlwyd]

I'm still wary of moving tests out of the default cargo test workflow -- folks, especially newer contributors, are likely to encounter mysterious failures in CI that they didn't see locally because they didn't run the tests locally. But let's see how it goes; we can always point folks at the README.

[divergentdave]

I'm concerned that we don't run cargo xtask test-docker anywhere in CI (unless I missed it). Maybe if test-docker were broken out into two subcommands, one for building the container images and one that uses them in a test, then the former could be used in the janus_interop_docker job and the latter could be used in janus_build?

Or if we ran cargo xtask test-docker in janus_build after loading the Docker images tarball emitted by janus_interop_docker, would docker buildx bake swiftly notice that the images it wants to build are already present in the local repository and move on, or would it insist on rebuilding them?

I added separate subcommands for just building container images and just running tests. I changed the CI to use cargo xtask test-docker-with-images on the consuming end. I think I'd prefer using docker/bake-action instead of cargo xtask build-docker on the producing end still, because that sets up the GitHub Actions caching backend for us, and I would still have to go through some other hoops to tag, save, and upload the images.

Getting rid of artifacts uploads/downloads and relying on layer caching might work, plus keeping the needs relationship between jobs would help with cache hit rates when things are rebuilt. I'll try that out in a bit.