roachtest: package external dependencies in a reproducible way

[golgeek]

As pointed out in this PR and comment, we currently don't have an easy way of packaging and hosting custom software dependencies used in roachtests. This leads to CPU time being used to compile external dependencies during each roachtest.

In a previous life, I wrote a Golang package that used yaml definition files to describe external repositories, build steps and .deb/.rpm packages contents to solve this exact issue in an automated and reproducible way.

As a random example to illustrate, running the tool with this file would have created a .deb file for Debian Bullseye and Bookworm and an .rpm for CentOS 8, and would have pushed the packages to a package repository:

repository:
  URL: "git@github.com:krallin/tini.git"

buildSteps:
  - |
    export CFLAGS="-DPR_SET_CHILD_SUBREAPER=36 -DPR_GET_CHILD_SUBREAPER=37"
    cmake . && make

packaging:
  - package: "tini"
    license: "MIT"
    architecture: "auto"
    files:
      - src: "./tini"
        dst: "/usr/bin/tini"

environments:
  - name: "debian:bookworm"
    template: "debian"
    build:
      dependencies:
        - type: "package::main"
          name: "build-essential"
        - type: "package::main"
          name: "cmake"

  - name: "debian:bullseye"
    template: "debian"
    build:
      dependencies:
        - type: "package::main"
          name: "build-essential"
        - type: "package::main"
          name: "cmake"

  - name: "centos:8"
    template: "centos"
    build:
      dependencies:
        - type: "package::main"
          name: "gcc"
        - type: "package::main"
          name: "gcc-c++"
        - type: "package::main"
          name: "glibc-devel"
        - type: "package::main"
          name: "glibc-static"
        - type: "package::main"
          name: "make"
        - type: "package::main"
          name: "cmake"

The tool also supported adding extra files as systemd services config files that would solve the messy start logic that we use right now.

I still have the code somewhere, which would need a good bit of adapting and refactoring, but a tool like this could allow us to just run add-apt-repository with a custom repository in the init script, and then apt install [PACKAGE] precompiled bits of software.

Jira issue: CRDB-51015

[blathers-crl]

cc @cockroachdb/test-eng