release: v0.3.0 - operations (unified CLI, cloud images, metrics, security baseline) - #2
Merged
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Comment on lines
+562
to
+700
| needs: build-cloud | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| include: | ||
| - arch: amd64 | ||
| runner: ubuntu-24.04 | ||
| # KVM available: inference is asserted | ||
| strict: "yes" | ||
| - arch: arm64 | ||
| runner: ubuntu-24.04-arm | ||
| # no KVM: TCG-emulated inference is informational only | ||
| strict: "no" | ||
| runs-on: ${{ matrix.runner }} | ||
| timeout-minutes: ${{ matrix.arch == 'arm64' && 150 || 45 }} | ||
| steps: | ||
| - name: Download cloud image | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| name: sibillaos-cloud-${{ matrix.arch }} | ||
| path: cloud-img | ||
| - name: Install QEMU | ||
| run: | | ||
| sudo apt-get update | ||
| if [ "${{ matrix.arch }}" = "arm64" ]; then | ||
| sudo apt-get install -y qemu-system-arm qemu-efi-aarch64 qemu-utils xorriso sshpass jq | ||
| else | ||
| sudo apt-get install -y qemu-system-x86 qemu-utils xorriso sshpass jq | ||
| fi | ||
| - name: Deploy and probe | ||
| run: | | ||
| IMG=$(ls cloud-img/*.qcow2 | head -1) | ||
| if [ "${{ matrix.arch }}" = "arm64" ]; then | ||
| QEMU=qemu-system-aarch64 | ||
| KVM="-machine virt -cpu max -bios /usr/share/AAVMF/AAVMF_CODE.fd" | ||
| if [ -e /dev/kvm ]; then | ||
| sudo chmod 666 /dev/kvm | ||
| KVM="-machine virt -enable-kvm -cpu host -bios /usr/share/AAVMF/AAVMF_CODE.fd" | ||
| fi | ||
| else | ||
| QEMU=qemu-system-x86_64 | ||
| KVM="" | ||
| if [ -e /dev/kvm ]; then | ||
| sudo chmod 666 /dev/kvm | ||
| KVM="-enable-kvm -cpu host" | ||
| fi | ||
| fi | ||
| # a deployer-style NoCloud seed: own user, password ssh | ||
| mkdir seed | ||
| cat > seed/user-data <<'EOF' | ||
| #cloud-config | ||
| hostname: sibilla-cloud | ||
| ssh_pwauth: true | ||
| users: | ||
| - name: sibilla | ||
| plain_text_passwd: sibilla | ||
| lock_passwd: false | ||
| shell: /bin/bash | ||
| groups: sudo | ||
| sudo: ALL=(ALL) NOPASSWD:ALL | ||
| EOF | ||
| printf 'instance-id: sibilla-cloud-ci\n' > seed/meta-data | ||
| xorriso -as mkisofs -volid CIDATA -joliet -rock -o seed.iso seed | ||
| # shellcheck disable=SC2086 | ||
| $QEMU $KVM -m 4096 -smp 4 \ | ||
| -display none -serial file:cloud.log \ | ||
| -drive "file=$IMG,format=qcow2,if=virtio" \ | ||
| -drive "file=seed.iso,format=raw,if=virtio,readonly=on" \ | ||
| -netdev user,id=n0,hostfwd=tcp::8080-:8080,hostfwd=tcp::2222-:22 \ | ||
| -device virtio-net,netdev=n0 & | ||
| QPID=$! | ||
| ok=0 | ||
| # first boot downloads the model; under TCG everything is | ||
| # slower, so allow up to 40 minutes | ||
| for _ in $(seq 1 240); do | ||
| sleep 10 | ||
| code=$(curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:8080/v1/models || true) | ||
| if [ "$code" = "401" ]; then | ||
| ok=1 | ||
| break | ||
| fi | ||
| done | ||
| RC=1 | ||
| if [ "$ok" -eq 1 ]; then | ||
| echo "gateway answered 401" | ||
| SSH="sshpass -p sibilla ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -p 2222 sibilla@127.0.0.1" | ||
| KEY=$($SSH "sudo cat /etc/llmd/apikey" | tail -1) | ||
| MODEL=$($SSH "sudo cat /etc/llmd/model" | tail -1) | ||
| echo "model: $MODEL" | ||
| if [ -n "$KEY" ] && [ -n "$MODEL" ]; then | ||
| RC=0 | ||
| # the API surface is asserted on every architecture | ||
| mjson=$(curl -s -H "Authorization: Bearer $KEY" http://127.0.0.1:8080/v1/models) || true | ||
| if echo "$mjson" | jq -e --arg m "$MODEL" '.data[] | select(.id == $m)' >/dev/null; then | ||
| echo "served models include $MODEL" | ||
| else | ||
| echo "served models do not include $MODEL: $mjson" | ||
| RC=1 | ||
| fi | ||
| if $SSH "sibilla status"; then | ||
| true | ||
| else | ||
| echo "sibilla status failed on the cloud image" | ||
| RC=1 | ||
| fi | ||
| # inference: asserted with KVM, informational under TCG | ||
| # (emulated token generation is minutes per sentence) | ||
| RESP=$(curl -s -m 1500 http://127.0.0.1:8080/v1/chat/completions \ | ||
| -H "Authorization: Bearer $KEY" \ | ||
| -H "Content-Type: application/json" \ | ||
| -d "{\"model\": \"$MODEL\", \"messages\": [{\"role\": \"user\", \"content\": \"Say ok.\"}], \"max_tokens\": 60}") || true | ||
| if echo "$RESP" | jq -e '.choices[0].message | ((.content // "") + (.reasoning // "")) | length > 0' >/dev/null; then | ||
| echo "CLOUD INFERENCE OK" | ||
| else | ||
| if [ "${{ matrix.strict }}" = "yes" ]; then | ||
| echo "cloud inference failed" | ||
| echo "$RESP" | jq . || echo "$RESP" | ||
| RC=1 | ||
| else | ||
| echo "inference not asserted on this architecture (no KVM, TCG emulation)" | ||
| echo "$RESP" | head -c 400 || true | ||
| echo | ||
| fi | ||
| fi | ||
| else | ||
| echo "could not read api key or model over ssh" | ||
| fi | ||
| else | ||
| echo "gateway never came up; console tail:" | ||
| tail -80 cloud.log || true | ||
| fi | ||
| kill "$QPID" 2>/dev/null || true | ||
| exit "$RC" | ||
| - name: Upload cloud debug artifacts | ||
| if: always() | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: cloud-debug-${{ matrix.arch }} | ||
| path: cloud.log |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings main up to v0.3.0, the operations release. Full notes in
docs/releases/v0.3.0.md; decision log updated in docs/architecture.md.
Scope
sibillaCLI as the single entry point (status, model,tls, metrics, webui, connect), with the sibilla-* commands kept as
aliases.
sibilla statusis a real health view (engine, servedmodels, disk, GPU, gateway) and exits nonzero on failure;
sibilla model rmandprunereclaim disk without touching the servedmodel.
published for amd64 and arm64. Deployers attach their own
cloud-init user-data; engine and model detection moved into
llmd-firstboot so it runs on the deployed hardware. The arm64
variant builds on GitHub's native arm64 runners, with the llmfit
deb repackaged from the aarch64 musl release.
sibilla metrics enableserves Prometheusmetrics at /metrics/gateway behind the same bearer token; Grafana
dashboard and scrape config in docs/observability/. vLLM's native
/metrics passes through; Ollama at the pinned 0.31.1 has none
(verified in source).
only, webui port closed by default, --acme opens 80/443 itself),
unattended security updates, systemd sandbox extended to caddy,
firstboot and both containers.
Notable fixes
renderer now emits the servers-metrics global option, and the site
body moved into a route block so the metrics endpoint can never be
served before the 401 check.
(string-form runcmd runs under dash, which killed the ERR trap),
with its output routed to the serial console.
and llmd-firstboot declare their real curl dependency.
Verification
The full pipeline is green: ISO build, BIOS and UEFI boot, unattended
install, inference, TLS and auth matrix, multi-model, webui plumbing,
metrics endpoint (authenticated, surviving TLS switches), firewall
and sandbox assertions, prune invariants; plus the cloud image built
and deployed with a real user seed on amd64 and arm64. Documented
limits: the arm64 deploy runs under TCG (no KVM on GitHub's arm64
runners), so its token generation is informational; the Open WebUI
interface and vLLM on physical GPUs stay outside CI.
After the squash merge, tag v0.3.0 on main: the release build attaches
the split ISO plus both cloud images with per-arch checksums.