HPC: Make load balancer available as CI-built statically linked binary

[linusseelinger]

This should make things easier to users, avoiding the compilation step.

We'll have to figure out how to distribute with / alongside hq, possibly a license question.

[Crambor]

HQ seems to have a very permissive MIT license thankfully! might be as simple as including a download script? I made a very simple one for x86 here:

#!/bin/bash

arch=$(uname -m)
if [ "$arch" = "x86_64" ]; then
    arch_suffix="x64"
else
    echo "Error: handling for architecture $arch unsupported"
    exit 1
fi

target_arch="linux-$arch_suffix"
url=$(curl -sL https://api.github.com/repos/It4innovations/hyperqueue/releases/latest | \
      grep -o "\"browser_download_url\": \"https://[^\"]*-${target_arch}.tar.gz\"" | \
      cut -d '"' -f 4)

filename="hq-${target_arch}.tar.gz"
[[ ! -f $filename ]] && wget $url -O $filename
[[ ! -f "hq" ]] && tar xzf $filename

[linusseelinger]

Could we just bundle a hq binary with ours? I guess (hq binary + load balancer binary) is a bit more reliable regarding user error etc. than (download script + load balancer binary)

[Crambor]

Agreed; we should then bundle specific (working and tested) binaries of HQ inside of the UM-Bridge releases, ideally linking #58 and #59 in the same PR for one nice big change

[linusseelinger]

Good plan! Would you like set this up? I think we could go with a CI run for each commit on master for now. Lev just opened #63 for CI testing the load balancer. We could test with exactly those binaries then